Amjad Arshad

Academic

Career

Verbatim text

VStorm
VStorm is a scheduling algorithm designed for processing big data. It optimizes processing speed and reduces delays by analyzing network traffic and matching tasks to the most suitable devices. By doing this, VStorm efficiently manages tasks, leading to better performance and faster completion of the job.
In this project, we developed a scheduler that is:
• Resource Aware
The scheduler will read the available resources first and then divide the jobs accordingly.
• Traffic-Aware
Monitors network traffic and performs rescheduling in case of bottleneck.
• Job Aware
The scheduler will is aware of the nature of the type of job in hand e.g. job is CPU bound or GPU bound
• Topology Aware
The scheduler reads the topology and maps jobs accordingly. Topology is usually a Job in the form of a Directed Acyclic Graph (DAG)
On top of this scheduler, we are providing a web app that shows the scheduler options to users. Allows them to submit jobs directly without opening multiple terminals. The general dashboard shows the experimental setup, throughput during the processing, and job statistics. VStorm in comparison with the Default, Resource Aware, and A3 increased the performance by 30 %.

VSTORM
TEAM:
Saad Ullah Khan 19I-0474
Haseeb Ramzan 19I-0475
Amjad Arshad 19I-0504
Supervisor Details:
Supervisor: Dr. Muhammad Aleem FAST NUCES Isb.
Co Supervisor: Mr. Asif Muhammad COMSATS Uni Isb.
OVERVIEW
VStorm is a dynamic scheduling algorithm for big data processing that maximizes throughput and reduces latency. It analyzes network traffic and adapts to the inherent structure of job, mapping tasks to suitable devices. This results in efficient task management and improved performance.
ARCHITECTURE
Job Submission
Job
Feature Extraction
ML Model
Job type identification
Job, Job Type
Interacts
Nimbus
Topology
Resource
Traffic
Job Type
Tune Scheduling
VSTORM
Supervisor Node - 1
Supervisor Node - N
Logger
TIMELINE
Sep-Oct Literature review and finalizing the research gap
Nov-Dec Designing solution and implementing topology & resource aware scheduling
Apr-May Implementing traffic aware & job type aware scheduling using ML model
Feb-Mar Testing the algorithm and providing the web interface
PROJECT FLOW
Topology Logical mapping of tasks
Resource Sorting slots on basis of system resources Physical mapping of tasks
Traffic Mapping on the basis of DAG Monitoring real-time traffic for rebalancing in case of bottleneck
Job Type Identifying job type using ML model Sorting slots on the basis of model output
TOOLS & TECHNOLOGIES
Technology Used:
Python, Java, MERN, Apache Storm Shell Scripting
Supervisor Name:
Dr. Muhammad Aleem Asif Muhammad(COMSATS Uni Isb.)
Group Members:
Saad Ullah Khan (i19 - 0474)
Haseeb Ramzan (i19 - 0475)
Amjad Arshad (i19 - 0504)

AI enrichment