YSU Big Data 05/02

Creator
Creator
Seonglae ChoSeonglae Cho
Created
Created
2024 May 2 4:49
Editor
Edited
Edited
2024 May 5 13:28
Refs
Refs

Questions to professor

  • Tell our plan to the professor and get feedback
  • vessel there is some limitations, migrate to AWS if too
  • Migrate to AWS from milestone 2 cause there are limitations to the vessel.vessel is virtualized
  • The minimum scope of the test is 2 fold
  • milestone 2’s minimum scope is just about system configuration’s explanation and data migration and some details about the data and maybe some future plans.
  • performance thingy is for milestone 3 (system and data focus, multimode will make the test look better (single node is enough), clearly explain the system in detail, benchmark short intro
 
 

Tasks

We might distribute tasks for each teammates
  • Research what should we put in the presentation and report
    • Decide what sections to put on the presentation
  • Experiments
 
 
 
Maybe we can divide the work into those 4 tasks
  • Define System Components (node, data ingestion, processing(training, validation, scoring), storage, etc
  • AWS Implementation ( S3, EC2, Lambda), I think this is relevant to data migration
we need AWS S3 to store the data, EC2 for computation-heavy tasks such as data processing, etc
  • Develop a Prototype
  • Testing
 
 
 
 
 
 

Recommendations