Overview
PYQ Analysis & Exam Strategy
Deep analysis of AKTU 2023-24 (KCS061, 100 marks) and 2024-25 (BCS061, 70 marks) papers — repeated topics, weightage, and scoring strategy.
2
PYQs
5
Units
24+
Sure-Shot Qs
5
Key Diagrams
70
Max Marks
⚠️
Paper changed in 2024-25: now 70 marks (was 100). Section C = 7 marks each (was 10). Write concisely and always include diagrams.
✅
Paper pattern: Section A = all 7 short 2-mark Qs, Section B = attempt 3 of 5 (7 marks each), Section C = attempt 1 of 2 per question (7 marks each, 5 questions).
💡
New in 2024-25: Master-Slave vs Peer-Peer Replication, Spark components, and detailed Pig comparisons appeared for the first time. Prepare as bonus topics.
Most Repeated Topics — Frequency Table
| Topic | 2023-24 | 2024-25 | Priority |
|---|---|---|---|
| 5 Vs of Big Data | ✅ 10 marks | ✅ 7 marks | 🔥 MUST DO |
| HDFS Architecture | ✅ 10+10 marks | ✅ 7 marks | 🔥 MUST DO |
| MapReduce Architecture | ✅ 10 marks | ✅ 7 marks | 🔥 MUST DO |
| Hadoop Ecosystem Components | ✅ 10 marks | ✅ Asked Twice | 🔥 MUST DO |
| YARN Architecture | ✅ 2 marks | ✅ 7 marks | 🔥 MUST DO |
| Apache Hive Architecture | ✅ 10 marks | ✅ 7 marks | 🔥 MUST DO |
| Hadoop in Cloud (AWS/Azure/GCP) | ✅ 10 marks | ✅ 7 marks | 🔥 MUST DO |
| Analysis vs Reporting | ✅ 10 marks | ✅ 7 marks | ⭐ Very Imp |
| Spark / RDDs | ✅ 10 marks | ✅ 7 marks | ⭐ Very Imp |
| Apache Pig / Pig Latin | ✅ 2+2 marks | ✅ 7 marks | ⭐ Very Imp |
| NoSQL Databases | ✅ 10 marks | — | Medium |
| MongoDB CRUD | ✅ 10 marks | — | Medium |
| HBase Architecture | ✅ 10 marks | — | Medium |
| Master-Slave / Peer-Peer Replication | — | ✅ 7 marks | 🆕 New 2025 |
| Types of Analytics | — | ✅ 7 marks | 🆕 New 2025 |
Unit-wise Weightage & Priority
Unit 1 Big Data Introduction
Weight: 20–25% · Priority: High
5 Vs, Analytics types, Analysis vs Reporting
5 Vs, Analytics types, Analysis vs Reporting
Unit 2 Hadoop & MapReduce
Weight: 25–30% · Priority: Highest
Ecosystem, MapReduce phases, Streaming
Ecosystem, MapReduce phases, Streaming
Unit 3 HDFS & Hadoop Env
Weight: 20–25% · Priority: High
HDFS Read/Write, NameNode/DataNode, Cloud
HDFS Read/Write, NameNode/DataNode, Cloud
Unit 4 YARN, NoSQL, Spark
Weight: 20–25% · Priority: High
YARN Architecture, RDDs, MongoDB CRUD
YARN Architecture, RDDs, MongoDB CRUD
Unit 5 Pig, Hive, HBase
Weight: 15–20% · Priority: High
Hive Architecture, Pig vs Hive, HBase
Hive Architecture, Pig vs Hive, HBase
AKTU Paper Pattern (2024-25)
| Section | Questions | Marks | Total | Strategy |
|---|---|---|---|---|
| Section A | Attempt ALL 7 short Qs | 2 each | 14 | Definition + 2 bullet points. Do this first — easy 14 marks. |
| Section B | Attempt any 3 of 5 | 7 each | 21 | Pick your strongest 3 topics. Always draw a diagram. |
| Section C (Q3–Q7) | Attempt 1 of 2 per question | 7 each | 35 | One topic per question pair. Diagrams are mandatory here. |
| Total | 70 | — | ||
Unit 1 · 6 Hours
Introduction to Big Data
The 5 Vs and Analysis vs Reporting are guaranteed questions — both appeared in both PYQs (7–10 marks each).
🔥
5 Vs of Big Data appeared in BOTH years — 10 marks in 2023-24 and 7 marks in 2024-25. The single highest-probability question in the entire Big Data paper. Know it cold.
Core Topics
🔥 5 Vs of Big Data Both Years
Volume, Velocity, Variety, Veracity, Value. Appears as Section A (2-mark), Section B, and Section C. The most repeated Big Data topic in AKTU history.
⭐ Analysis vs Reporting Both Years
Advanced analytics uncovers hidden patterns (predictive); reporting summarises known data (descriptive). 7–10 mark question in both papers.
Types of Digital Data Short 2-mark
Structured: SQL tables, Excel sheets.
Semi-structured: JSON, XML, CSV.
Unstructured: Images, videos, emails, social media posts.
Semi-structured: JSON, XML, CSV.
Unstructured: Images, videos, emails, social media posts.
Big Data Platforms Short 2-mark
Hadoop, Apache Spark, Apache Kafka, Apache Flink, Google BigQuery, Amazon Redshift, Apache Storm, Azure HDInsight.
5 Vs — Complete Revision Table
| V | Name | Meaning | Real-World Example |
|---|---|---|---|
| V₁ | Volume | Massive quantity of data generated every second | Facebook: 500TB/day; YouTube: 500 hrs video/min uploaded |
| V₂ | Velocity | Speed at which data is generated and must be processed | Twitter: 6,000 tweets/sec; Stock exchange: microsecond feeds |
| V₃ | Variety | Multiple data types and formats from diverse sources | Text, images, video, audio, sensor data, log files |
| V₄ | Veracity | Quality, accuracy and trustworthiness of data | Noisy IoT sensors; duplicate records; fake social media accounts |
| V₅ | Value | Actionable business insight extracted after processing | Netflix recommendations; Amazon product suggestions; fraud detection |
🧠 Memory Trick: "Very Vast Varieties Verify Value" → V · V · V · V · V
Analysis vs Reporting — Comparison Table
| Feature | Data Analysis | Reporting |
|---|---|---|
| Purpose | Discover hidden patterns & insights | Summarise and present known facts |
| Approach | Exploratory, hypothesis-driven | Descriptive, structured, periodic |
| Output | Actionable insights, predictions, models | Charts, tables, dashboards, KPIs |
| Techniques | ML, statistical modelling, data mining | SQL queries, pivot tables, aggregation |
| Users | Data scientists, ML engineers | Business managers, executives |
| Example Question | Why are customers churning? | How many customers left this month? |
4 Types of Big Data Analytics New 2024-25
📊 Descriptive Analytics
Answers: What happened?
Summarises historical data. Example: Monthly sales report, website traffic dashboard, page view counts.
Summarises historical data. Example: Monthly sales report, website traffic dashboard, page view counts.
🔍 Diagnostic Analytics
Answers: Why did it happen?
Identifies root causes through drill-down. Example: Why did Q3 sales drop? Root cause analysis.
Identifies root causes through drill-down. Example: Why did Q3 sales drop? Root cause analysis.
🔮 Predictive Analytics
Answers: What will happen?
Uses ML and statistical models. Example: Customer churn prediction, demand forecasting, weather prediction.
Uses ML and statistical models. Example: Customer churn prediction, demand forecasting, weather prediction.
⚙️ Prescriptive Analytics
Answers: What should we do?
Recommends actions and decisions. Example: Optimal pricing engine, supply chain optimisation, route planning.
Recommends actions and decisions. Example: Optimal pricing engine, supply chain optimisation, route planning.
📌 Unit 1 — Quick Revision & 5 Vs Diagram
5 Vs Pentagon — Draw This in Exam for Extra Marks
Fig 1.1 — 5 Vs of Big Data (Pentagon Model) — Label each circle with name + brief meaning
🔴 Must-Know Definitions
- Volume: Scale of data (PB, EB level)
- Velocity: Speed of data generation & processing
- Variety: Structured / semi / unstructured types
- Veracity: Data quality & accuracy
- Value: Business insights extracted from data
⭐ Expected 2-Mark Answers
- List 5 Big Data platforms
- Structured vs Unstructured data examples
- What is Big Data? (definition)
- What is data analytics?
- What is Veracity in Big Data?
🏆 High-Mark Focus Topics
- 5 Vs full explanation with diagram (7–10 marks)
- Analysis vs Reporting comparison table (7–10 marks)
- Real-world Big Data applications
- 4 types of Big Data analytics
Unit 1 — Expected Questions
[Q1 SURE SHOT] Describe the 5 Vs of Big Data. What does each represent and why are they essential for data management? Repeated Both Years
[Q2 VERY IMP] Distinguish between data analysis and reporting in Big Data. How does advanced analytics uncover hidden patterns beyond traditional reporting? Both Years
[Q3] Discuss the various types of analytics used in Big Data (Descriptive, Diagnostic, Predictive, Prescriptive). New 2024-25
[Q4] Provide real-world Big Data examples in healthcare, finance, e-commerce, and transportation industries.
[Short 2-mark] What are structured, semi-structured, and unstructured data? Give examples of each.
[Short 2-mark] List any five Big Data platforms.
Unit 2 · 8 Hours
Hadoop & MapReduce
Highest-weightage unit. Hadoop ecosystem components and MapReduce architecture appear in every AKTU paper. "Hadoop ecosystem" was asked twice in 2024-25.
🔥
"Hadoop Ecosystem Components" appeared TWICE in the 2024-25 paper (Q1e and Q4b). Prepare a complete 11-component table and reproduce it from memory.
Hadoop Ecosystem — All 11 Components
| Component | Category | Purpose |
|---|---|---|
| HDFS | Storage | Distributed file system — stores data reliably across cluster nodes in 128 MB blocks |
| MapReduce | Processing | Distributed batch processing framework (Map → Shuffle → Reduce) |
| YARN | Resource Mgmt | Manages CPU/RAM; schedules and monitors all jobs on the cluster |
| Hive | Query | SQL-like querying (HiveQL) on Hadoop — used by data analysts |
| Pig | Scripting | High-level ETL scripting using Pig Latin; translates to MapReduce jobs |
| HBase | Database | Column-oriented NoSQL database on top of HDFS — random read/write |
| ZooKeeper | Coordination | Distributed coordination and synchronisation service for the cluster |
| Sqoop | Ingestion | Transfers structured data between RDBMS ↔ Hadoop (bulk import/export) |
| Flume | Ingestion | Collects and moves streaming/log data into HDFS in real time |
| Spark | Processing | Fast in-memory processing engine — up to 100× faster than MapReduce |
| Oozie | Workflow | Job scheduling and workflow coordination for Hadoop job pipelines |
MapReduce — Step by Step
1
Input Splits: Input data in HDFS is divided into fixed-size chunks (default 128 MB). Each split is processed in parallel by a separate Mapper task.
2
Map Phase: Mapper processes each record. Input: (key, value) pairs. Output: intermediate (key, value) pairs. Example: word → (word, 1).
3
Combiner (Optional): Mini-reducer running locally on each mapper node. Reduces network traffic before shuffle. Acts as local pre-aggregation.
4
Shuffle & Sort: Framework sorts all intermediate pairs by key and transfers to the appropriate Reducer. Same key always goes to same Reducer.
5
Reduce Phase: Reducer aggregates all values for a given key into a single output value. Example: (word,[1,1,1]) → (word,3).
6
Output to HDFS: Each Reducer writes one output file to HDFS. Client reads the final results from these files.
Classic Word Count Example
Input: "Big Data is Big and Big Data is Fast"
-- Map Phase --
(Big,1) (Data,1) (is,1) (Big,1) (and,1) (Big,1) (Data,1) (is,1) (Fast,1)
-- After Shuffle & Sort (grouped by key) --
(Big,[1,1,1]) (Data,[1,1]) (Fast,[1]) (and,[1]) (is,[1,1])
-- Reduce Output --
(Big,3) (Data,2) (Fast,1) (and,1) (is,2)
-- Map Phase --
(Big,1) (Data,1) (is,1) (Big,1) (and,1) (Big,1) (Data,1) (is,1) (Fast,1)
-- After Shuffle & Sort (grouped by key) --
(Big,[1,1,1]) (Data,[1,1]) (Fast,[1]) (and,[1]) (is,[1,1])
-- Reduce Output --
(Big,3) (Data,2) (Fast,1) (and,1) (is,2)
Hadoop Streaming — Short Note
Hadoop Streaming is a utility that allows developers to write MapReduce programs in any language that can read from stdin and write to stdout — Python, Ruby, Perl, or bash scripts.
The Streaming API creates Map and Reduce tasks as standard processes, then pipes data in/out via stdin/stdout.
Key Advantage: No need to write Java. Data scientists can use Python directly to write MapReduce jobs.
Usage:
The Streaming API creates Map and Reduce tasks as standard processes, then pipes data in/out via stdin/stdout.
Key Advantage: No need to write Java. Data scientists can use Python directly to write MapReduce jobs.
Usage:
hadoop jar streaming.jar -mapper mapper.py -reducer reducer.py -input /data -output /result
📌 Unit 2 — Quick Revision & MapReduce Diagram
MapReduce Architecture — Draw This in Exam
Fig 2.1 — MapReduce Architecture (Input → Splits → Map → Shuffle & Sort → Reduce → Output to HDFS)
🔴 Key Facts to Memorise
- Default split size = 128 MB
- JobTracker manages jobs (Hadoop 1.x)
- TaskTracker runs tasks on each node
- YARN replaces JobTracker in Hadoop 2.x
- Combiner is optional — acts as local Reducer
⚠️ Common Exam Mistakes
- MapReduce is a processing framework — NOT a database
- Combiner is not always present in MapReduce
- Same key always goes to the same Reducer (after shuffle)
- Shuffle & Sort is done by the framework, not the developer
Unit 2 — Expected Questions
[Q1 SURE SHOT] Explain Apache Hadoop and its role in Big Data processing. What are the core components of the Hadoop ecosystem and how do they work together? Both Years — asked twice in 2024-25
[Q2 SURE SHOT] Discuss the detailed architecture of MapReduce — Map phase, Shuffle & Sort, Reduce phase — with a diagram. Both Years
[Q3 VERY IMP] Discuss the process of developing a MapReduce application. What are the key steps in writing, testing, and deploying?
[Short 2-mark] What is Hadoop Streaming?
[Short 2-mark] Discuss data formats commonly used in Hadoop (SequenceFile, Avro, Parquet, ORC, Text).
[Short 2-mark] How does Apache Hadoop help process data?
Unit 3 · 8 Hours
HDFS & Hadoop Environment
HDFS architecture and read/write operations appear in every paper. Cloud deployment appeared in both PYQs. The NameNode vs Secondary NameNode distinction is a very common trap.
⚠️
Most common exam trap: Secondary NameNode is NOT a backup of NameNode! It only periodically checkpoints metadata (merges fsimage + edit logs). If you write this incorrectly, marks will be deducted.
HDFS Architecture — All Components
🗄️ NameNode (Master)
• Stores all metadata (file names, block locations, permissions, directory structure)
• Does NOT store actual file data — only the directory tree
• Single point of failure → solved by HA NameNode in Hadoop 2.x
• Maintains fsimage (filesystem snapshot) and edit logs (change transactions)
• Runs on the master node of the cluster
• Does NOT store actual file data — only the directory tree
• Single point of failure → solved by HA NameNode in Hadoop 2.x
• Maintains fsimage (filesystem snapshot) and edit logs (change transactions)
• Runs on the master node of the cluster
💾 DataNode (Slave)
• Stores actual data blocks on local disk
• Sends heartbeat to NameNode every 3 seconds (proof of life)
• Sends block report to NameNode every 6 hours
• Handles read/write requests directly from clients
• Reports full block list to NameNode on startup
• Sends heartbeat to NameNode every 3 seconds (proof of life)
• Sends block report to NameNode every 6 hours
• Handles read/write requests directly from clients
• Reports full block list to NameNode on startup
🔄 Secondary NameNode
• NOT a backup — it is a checkpoint node only
• Periodically merges fsimage + edit logs (checkpointing process)
• Reduces NameNode restart time after failure
• Runs on a separate machine from primary NameNode
• Periodically merges fsimage + edit logs (checkpointing process)
• Reduces NameNode restart time after failure
• Runs on a separate machine from primary NameNode
📦 Block Replication
• Default block size = 128 MB
• Default replication factor = 3 (configurable)
• Rack-aware placement: 2 replicas on same rack, 1 on different rack
• If a DataNode fails, data is still available from other replicas
• Default replication factor = 3 (configurable)
• Rack-aware placement: 2 replicas on same rack, 1 on different rack
• If a DataNode fails, data is still available from other replicas
HDFS Read Operation (5 Steps)
1
Client → NameNode: Client opens the file by calling open() on DistributedFileSystem with the file path.
2
NameNode Response: NameNode returns block locations — IP addresses of DataNodes containing each block, sorted by proximity to the client.
3
Client → Nearest DataNode: Client connects to the closest DataNode using FSDataInputStream and reads the first block.
4
Checksum Verification: Client verifies data integrity using checksums stored with each block. If corrupted, reads from next replica automatically.
5
Repeat & Assemble: Client reads all blocks sequentially. If a DataNode fails mid-read, moves to the next replica. All blocks assembled to reconstruct the complete file.
HDFS Write Operation (7 Steps)
1
Client → NameNode: Client calls create() on DistributedFileSystem. NameNode checks permissions and ensures the file doesn't already exist.
2
DataNode Assignment: NameNode selects 3 DataNodes using rack-aware algorithm and returns their addresses to the client.
3
Pipeline Setup: DataNodes form a replication pipeline: Client → DataNode₁ → DataNode₂ → DataNode₃.
4
Data Packets: Client breaks data into packets and writes to DataNode₁ through FSDataOutputStream.
5
Replication: DataNode₁ replicates to DataNode₂; DataNode₂ replicates to DataNode₃ — simultaneously down the pipeline.
6
Acknowledgments: Each DataNode sends an ACK back up the pipeline to the client confirming successful receipt of the packet.
7
Close & Notify NameNode: Client closes the file. NameNode updates metadata with final block locations and marks the file as complete.
Hadoop in Cloud — Appeared Both Years
✅ Advantages
• No upfront hardware cost (pay-as-you-go)
• Auto-scaling based on workload demand
• Managed services reduce administration overhead
• Global availability & built-in disaster recovery
• Integration with native cloud storage (S3, Blob, GCS)
• Auto-scaling based on workload demand
• Managed services reduce administration overhead
• Global availability & built-in disaster recovery
• Integration with native cloud storage (S3, Blob, GCS)
⚠️ Challenges
• Data transfer latency and bandwidth costs
• Security and compliance concerns (data sovereignty)
• Vendor lock-in risk when using proprietary features
• Unpredictable billing for variable workloads
• Network dependency for distributed computation
• Security and compliance concerns (data sovereignty)
• Vendor lock-in risk when using proprietary features
• Unpredictable billing for variable workloads
• Network dependency for distributed computation
| Cloud Provider | Managed Hadoop Service | Cloud Storage |
|---|---|---|
| Amazon Web Services | Amazon EMR (Elastic MapReduce) | Amazon S3 |
| Microsoft Azure | Azure HDInsight | Azure Blob Storage |
| Google Cloud Platform | Google Cloud Dataproc | Google Cloud Storage |
📌 Unit 3 — Quick Revision & HDFS Architecture Diagram
HDFS Architecture — Most Important Diagram in the Entire Course
Fig 3.1 — HDFS Architecture (Client, NameNode, Secondary NameNode, 3 DataNodes with Rack Awareness)
🔴 Critical Facts
- Block size = 128 MB (default)
- Replication factor = 3 (default)
- Secondary NameNode = Checkpoint only — NOT backup
- DataNode heartbeat = every 3 seconds
- DataNode declared dead if no heartbeat for 10 minutes
☁️ Cloud Platforms (Both Years!)
- AWS → Amazon EMR
- Azure → Azure HDInsight
- GCP → Cloud Dataproc
- Key advantage: Auto-scaling
- Key challenge: Data transfer latency
Unit 3 — Expected Questions
[Q1 SURE SHOT] Explain HDFS — NameNode, DataNode, file system namespace. How does HDFS manage storage and replication across a distributed cluster? Both Years
[Q2 SURE SHOT] Explain how HDFS stores, reads, and writes files. Describe the complete sequence of operations for each process. Both Years
[Q3 VERY IMP] Discuss advantages and challenges of deploying Hadoop in cloud (AWS EMR / Azure HDInsight / GCP Dataproc). Both Years
[Q4] Discuss cluster specification. Describe how to set up a Hadoop Cluster step by step. New 2024-25
[Short 2-mark] What is HDFS in the context of Hadoop?
[Short 2-mark] What are block sizes and block abstraction in HDFS?
[Short 2-mark] What are the benefits and challenges of HDFS for distributed storage?
Unit 4 · 9 Hours
YARN, NoSQL, Spark & MongoDB
YARN architecture and Spark RDDs appeared in both PYQs. NoSQL and MongoDB are common Section B choices. Scala is low-priority for AKTU.
YARN Architecture — All 4 Components
| Component | Runs Where | Detailed Role |
|---|---|---|
| ResourceManager | 1 per cluster (master node) | Global resource allocator. Two sub-components: Scheduler (allocates resources to apps) and ApplicationsManager (accepts job submissions, restarts failed AppMasters). |
| NodeManager | Every worker node | Per-node agent. Monitors CPU/RAM usage on that node, manages containers, reports resource availability to ResourceManager, kills containers as instructed. |
| ApplicationMaster | 1 per application (worker node) | Per-application coordinator. Negotiates resources with ResourceManager, works with NodeManager to execute tasks, monitors task progress, handles task failures. |
| Container | Worker nodes | Basic unit of resource allocation (specific CPU + RAM amount). All actual computation tasks (Mappers, Reducers, Spark executors) run inside containers. |
💡
Why YARN matters: In Hadoop 1.x, MapReduce handled BOTH resource management AND processing — limiting the cluster to MapReduce only. YARN separates these, allowing Spark, Hive, Pig, and more to share the same cluster (multi-tenancy).
YARN Schedulers Short 2-mark
⚖️ Fair Scheduler
Resources shared equally among all running applications. Short jobs finish quickly. Good for mixed small/large workloads. Default in CDH (Cloudera).
📊 Capacity Scheduler
Cluster resources divided into queues with guaranteed capacity per team/department. Unused capacity shared with others. Default in Apache Hadoop (HDP).
Apache Spark & RDDs
What is an RDD?
Resilient = Fault-tolerant (recomputes lost partitions from lineage)
Distributed = Data spread across cluster nodes in partitions
Dataset = Immutable collection of elements processed in parallel
Distributed = Data spread across cluster nodes in partitions
Dataset = Immutable collection of elements processed in parallel
Why Spark is 100× faster?
MapReduce reads/writes to disk after every step (slow I/O).
Spark stores intermediate results in RAM between steps.
Critical for iterative ML algorithms that run 100+ passes over data.
Spark stores intermediate results in RAM between steps.
Critical for iterative ML algorithms that run 100+ passes over data.
| Operation Type | Examples | When Executed? | Returns |
|---|---|---|---|
| Transformations (Lazy) | map(), filter(), flatMap(), groupByKey(), reduceByKey() | NOT immediately — only when an action is called | New RDD |
| Actions (Eager) | count(), collect(), reduce(), saveAsTextFile(), first() | Immediately triggers full computation on the cluster | Value or side-effect |
NoSQL Database Types
| Type | Storage Model | Examples | Best Use Case |
|---|---|---|---|
| Key-Value | Simple key → value pairs | Redis, DynamoDB, Riak | Caching, session storage, shopping carts |
| Document | JSON / BSON documents | MongoDB, CouchDB, Firestore | Catalogs, user profiles, content management |
| Column-Family | Rows + column families (wide columns) | HBase, Cassandra, Bigtable | Time-series, IoT sensor data, analytics |
| Graph | Nodes + Edges + Properties | Neo4j, Amazon Neptune | Social networks, fraud detection, recommendations |
MongoDB CRUD Operations
| Operation | Command | Example Syntax |
|---|---|---|
| Create | insertOne() / insertMany() | db.students.insertOne({name:"Raj", marks:85, dept:"CS"}) |
| Read | find() / findOne() | db.students.find({marks:{$gt:80}}) |
| Update | updateOne() / updateMany() | db.students.updateOne({name:"Raj"},{$set:{marks:90}}) |
| Delete | deleteOne() / deleteMany() | db.students.deleteOne({name:"Raj"}) |
📌 Unit 4 — Quick Revision & YARN Diagram
YARN Architecture — Draw This in Exam
Fig 4.1 — YARN Architecture (Client → ResourceManager → NodeManagers → Containers)
⚡ RDD Quick Facts
- RDD = Resilient Distributed Dataset
- Transformations are lazy — deferred until action
- Actions are eager — trigger computation
- Spark uses DAG (Directed Acyclic Graph) for execution planning
- RDD lineage allows recomputation of lost partitions
🗄️ NoSQL vs RDBMS
- NoSQL = schema-less; RDBMS = fixed schema
- NoSQL = horizontal scaling; RDBMS = vertical
- NoSQL = BASE model; RDBMS = ACID
- 4 types: Key-Value, Document, Column-Family, Graph
Unit 4 — Expected Questions
[Q1 SURE SHOT] Discuss the detailed architecture of YARN — ResourceManager, NodeManager, ApplicationMaster, Container — with diagram. Both Years
[Q2 SURE SHOT] What are RDDs in Spark? How do they enable fault-tolerant distributed data processing? Both Years
[Q3 VERY IMP] Define NoSQL databases. What are the key characteristics and benefits compared to traditional relational databases?
[Q4 VERY IMP] Explain MongoDB CRUD operations with syntax and examples for each (insertOne, find, updateOne, deleteOne).
[Q5] Explain the features and components of Apache Spark (Core, SQL, Streaming, MLlib, GraphX). New 2024-25
[Short 2-mark] What are the characteristics of Fair Scheduler and Capacity Scheduler?
[Short 2-mark] What is the role of YARN in Hadoop?
Unit 5 · 9 Hours
Pig, Hive & HBase
Hive architecture appeared in both PYQs. 2024-25 introduced a direct three-way Pig comparison. Know Pig vs Hive vs MapReduce differences clearly with a table.
Apache Hive Architecture — Data Flow (7 Steps)
1
User writes HiveQL: SQL-like query written by data analyst. Example:
SELECT dept, COUNT(*) FROM employees GROUP BY dept;2
Driver: Receives the HiveQL query and manages its complete lifecycle — parsing, planning, execution, result delivery.
3
Compiler: Parses HiveQL → checks syntax → generates Abstract Syntax Tree (AST) → creates logical execution plan.
4
Metastore: Contains all table schemas, column types, partition info, and data locations in HDFS. Compiler fetches schema from here during planning.
5
Optimizer: Optimises the execution plan — partition pruning, predicate pushdown, join reordering — to minimise data read and shuffle.
6
Execution Engine: Converts the optimised plan into MapReduce / Tez / Spark jobs and submits them to YARN for execution on the cluster.
7
HDFS & Results: Jobs execute on the cluster. Data read from HDFS, processed, and results returned to the user via Hive CLI, Beeline, or JDBC/ODBC.
Pig vs Hive vs MapReduce vs SQL — Master Comparison Direct 2024-25 Question
| Feature | Apache Pig | Apache Hive | MapReduce | SQL (RDBMS) |
|---|---|---|---|---|
| Language | Pig Latin | HiveQL (SQL-like) | Java API | Standard SQL |
| Target Users | ETL developers, data engineers | Data analysts, SQL users | Java developers | SQL developers, DBAs |
| Schema | Optional (flexible, schema-on-read) | Required (strict, schema-on-read) | None — raw data | Required (schema-on-write) |
| Data Structures | Bags, Tuples, Maps | Tables, Rows, Columns | Key-value pairs | Tables, Rows, Columns |
| Ease of Use | Medium (scripting language) | Easy (familiar SQL syntax) | Hard (verbose Java code) | Easy (declarative SQL) |
| Data Exploration | Good | Excellent | Poor (very tedious) | Limited to DB size |
| Translates to | MapReduce jobs | MapReduce / Tez / Spark | Direct execution | Query engine (InnoDB etc.) |
| Best for | ETL pipelines, data transformation | Ad-hoc queries, BI reports | Custom batch processing | OLTP, transactional systems |
| Data Scale | Petabyte-scale (HDFS) | Petabyte-scale (HDFS) | Petabyte-scale | Gigabyte scale (single server) |
HBase Architecture
🏛️ HMaster
• Master node of HBase cluster
• Manages and monitors RegionServers
• Handles DDL operations (create, delete, alter tables)
• Performs load balancing across RegionServers
• Assigns Regions to RegionServers on startup/failure
• Manages and monitors RegionServers
• Handles DDL operations (create, delete, alter tables)
• Performs load balancing across RegionServers
• Assigns Regions to RegionServers on startup/failure
⚙️ RegionServer
• Handles read and write operations for data regions
• A Region = contiguous range of rows sorted by row key
• Each RegionServer manages multiple Regions
• Communicates with HDFS DataNodes for actual data storage
• A Region = contiguous range of rows sorted by row key
• Each RegionServer manages multiple Regions
• Communicates with HDFS DataNodes for actual data storage
| Feature | HBase | Traditional RDBMS |
|---|---|---|
| Data Model | Column-family (wide, sparse tables) | Row-based (normalized, dense tables) |
| Schema | Schema-less columns (highly flexible) | Fixed schema (ALTER TABLE needed to add columns) |
| Scalability | Horizontal (add more commodity nodes) | Vertical (upgrade server hardware — expensive) |
| Transactions | Single-row ACID only | Full multi-row, multi-table ACID transactions |
| Query Language | Java API / HBase Shell / REST API | Standard SQL with full JOIN support |
| Best for | Billions of rows, random real-time access | Complex queries, reporting, foreign keys |
| Real Examples | Facebook Messages, Twitter timelines, IoT data | Banking systems, e-commerce orders, inventory |
📌 Unit 5 — Quick Revision & Hive Architecture Diagram
Apache Hive Data Flow — Draw This in Exam
Fig 5.1 — Hive Architecture & Data Flow (HiveQL → Driver → Compiler ↔ Metastore → Optimizer → Executor → HDFS → Result)
🐷 Pig Key Facts
- Language = Pig Latin
- Interactive shell = Grunt
- Execution modes: Local, MapReduce, Tez
- Main ops: LOAD, FILTER, FOREACH, GROUP, JOIN, STORE
- Good for: ETL pipelines, data transformation at scale
🏛️ HBase Key Facts
- Column-family NoSQL DB built on HDFS
- Master = HMaster
- Workers = RegionServers
- Good for: billions of rows, random real-time access
- Examples: Facebook Messages, Twitter timelines
🐝 Hive Key Facts
- Language = HiveQL (SQL-like)
- Schema info stored in Metastore
- Translates to MapReduce / Tez / Spark
- Best for: ad-hoc SQL queries on large HDFS datasets
- Not good for: real-time queries (batch only)
Unit 5 — Expected Questions
[Q1 SURE SHOT] Provide an overview of Apache Hive architecture. How does Hive translate HiveQL queries into MapReduce jobs? Include data flow diagram. Both Years
[Q2 SURE SHOT] Compare and contrast: (i) Apache Pig vs MapReduce (ii) Pig vs SQL (iii) Pig vs Hive. 2024-25 direct question
[Q3 VERY IMP] Explain HBase — how it differs from relational databases and what advantages it offers for large-scale data storage and access.
[Q4 VERY IMP] Explain the role of Pig Latin as a high-level scripting language for processing large datasets. How does it simplify data transformation compared to MapReduce?
[Short 2-mark] What is Apache Pig?
[Short 2-mark] Describe the Grunt shell in Apache Pig.
[Short 2-mark] How does Hive differ from Pig in the Hadoop ecosystem?
Sure-Shot Questions
Top 24 Questions by Probability
Ranked by AKTU frequency, marks value, and trend analysis across PYQs 2023-24 and 2024-25.
🔥
TIER 1 — Appeared in BOTH papers. Prepare all 9 with complete diagrams and keyword-rich answers. Knowing even 5 of these well = 35–50+ marks.
01 Describe the 5 Vs of Big Data — Volume, Velocity, Variety, Veracity, Value — with examples. Why are they essential for data management and analysis? Both Years · 7–10 marks
02 Explain HDFS — NameNode, DataNode, file system namespace. How does HDFS manage data storage and replication across a distributed cluster? Both Years · 7–10 marks
03 Explain how HDFS stores, reads, and writes files. Describe the complete sequence of operations for each process. Both Years · 7–10 marks
04 Discuss the detailed architecture of MapReduce — Map phase, Shuffle & Sort, Reduce phase — with a clearly labelled diagram. Both Years · 7–10 marks
05 Explain Apache Hadoop and its role in Big Data processing. What are the core components of the Hadoop ecosystem and how do they work together? Both Years — asked twice in 2024-25 · 7–10 marks
06 Discuss the detailed architecture of YARN — ResourceManager, NodeManager, ApplicationMaster, Container — with a diagram. Both Years · 7–10 marks
07 Provide an overview of Apache Hive architecture. How does HiveQL translate SQL-like queries into MapReduce jobs? Include the complete data flow diagram. Both Years · 7–10 marks
08 Discuss advantages and challenges of deploying Hadoop in cloud (AWS EMR, Azure HDInsight, GCP Dataproc). How do cloud platforms enhance scalability and cost-efficiency? Both Years · 7–10 marks
09 Distinguish between data analysis and reporting in Big Data. How does advanced analytics go beyond traditional reporting to uncover hidden patterns and correlations? Both Years · 7–10 marks
⭐
TIER 2 — High Probability. Prepare complete answers. High chance of appearing in Section B or Section C.
10 What are RDDs in Apache Spark? How do they enable fault-tolerant and distributed data processing? Compare transformations vs actions. Both Years · 7–10 marks
11 Compare and contrast: (i) Apache Pig vs MapReduce (ii) Pig vs SQL (iii) Pig vs Hive — using a detailed comparison table. 2024-25 direct question
12 Define NoSQL databases. What are the key characteristics, four types, and benefits compared to traditional relational databases?
13 Explain MongoDB CRUD operations — insertOne, find, updateOne, deleteOne — with syntax examples and practical use.
14 Introduce HBase and its role in the Hadoop ecosystem. How does HBase differ from RDBMS and what advantages does it offer for large-scale data?
15 Discuss the process of developing a MapReduce application. What are the key steps in writing, testing, and deploying?
16 Discuss cluster specification. Describe how to set up a Hadoop Cluster step by step. New 2024-25
17 Explain the features and components of Apache Spark (Core, SQL, Streaming, MLlib, GraphX, Spark on YARN). New 2024-25
18 Discuss Master-Slave and Peer-Peer replication in detail with diagrams and use cases. New 2024-25
✅
TIER 3 — Short Notes (2-mark). Prepare crisp 40–60 word answers in bullet-point format. Do not skip these — they are 14 free marks.
19 Types of digital data — structured, semi-structured, unstructured — with at least one example of each.
20 What is YARN? — definition and 3 main components (ResourceManager, NodeManager, ApplicationMaster) in brief.
21 What is Apache Pig? Describe the Grunt shell and its purpose.
22 What are block sizes and block abstraction in HDFS? (128MB default, replication factor 3)
23 What is Hadoop Streaming? How does it allow non-Java programs to be used as MapReduce mappers/reducers?
24 What are the main components of the Hadoop ecosystem? List at least six with one-line role descriptions.
Model Answers
AKTU-Style Answer Writing Guide
How to structure answers, keywords to underline, diagrams to draw, and common mistakes to avoid in the AKTU Big Data exam.
Answer Length & Format Guide
| Marks | Word Count | Time | Required Structure |
|---|---|---|---|
| 2 marks | 40–60 words | 3–4 min | One definition sentence + 2–3 concise bullet points. No diagram needed unless architecture. |
| 7 marks | 250–350 words | 12–15 min | Introduction → Definition → Explanation (bullets) → Labelled Diagram → Conclusion |
| 10 marks | 400–500 words | 18–20 min | Introduction → Concept → Detailed Components → Diagram → Real Examples → Conclusion |
Model 2-Mark Answer: "What is YARN?"
YARN (Yet Another Resource Negotiator) is the resource management framework of Hadoop 2.x. It decouples resource management from data processing, allowing Spark, Hive, and MapReduce to share one cluster simultaneously (multi-tenancy).
Key components:
• ResourceManager — global resource allocator (master)
• NodeManager — per-node resource monitor and container launcher
• ApplicationMaster — per-application coordinator and task monitor
• Container — unit of resource allocation (specific CPU + RAM)
Key components:
• ResourceManager — global resource allocator (master)
• NodeManager — per-node resource monitor and container launcher
• ApplicationMaster — per-application coordinator and task monitor
• Container — unit of resource allocation (specific CPU + RAM)
Model 7-Mark Answer Structure: "5 Vs of Big Data"
[Para 1 — Introduction]
Big Data refers to extremely large, complex datasets that cannot be efficiently stored, processed, or analysed using traditional database management systems. The 5 Vs are the defining characteristics that guide Big Data system architecture and design decisions.
[Para 2 — Each V with definition + example]
1. Volume — Massive quantity of data generated continuously. Facebook: 500TB/day; YouTube: 500 hours of video uploaded per minute.
2. Velocity — Speed at which data is generated and must be processed. Twitter: 6,000 tweets/sec; Stock market: microsecond-level trade data.
3. Variety — Multiple data types from diverse sources: Structured (SQL tables), semi-structured (JSON/XML), unstructured (images, videos, audio).
4. Veracity — Quality, accuracy and trustworthiness of data. IoT sensor noise, duplicate records, and inconsistent social media posts challenge data quality.
5. Value — Actionable business insights extracted from processed data. Netflix's recommendation engine drives 80% of all viewing via data-derived suggestions.
[Diagram — Draw the Pentagon model with Big Data in center, V1–V5 at corners. Takes 60 seconds and earns 1–2 extra marks.]
[Conclusion]
Understanding the 5 Vs is critical for designing appropriate storage (Volume), streaming pipelines (Velocity), data lake architecture (Variety), data quality frameworks (Veracity), and analytics solutions (Value). Together, they define the full scope of Big Data challenges.
Big Data refers to extremely large, complex datasets that cannot be efficiently stored, processed, or analysed using traditional database management systems. The 5 Vs are the defining characteristics that guide Big Data system architecture and design decisions.
[Para 2 — Each V with definition + example]
1. Volume — Massive quantity of data generated continuously. Facebook: 500TB/day; YouTube: 500 hours of video uploaded per minute.
2. Velocity — Speed at which data is generated and must be processed. Twitter: 6,000 tweets/sec; Stock market: microsecond-level trade data.
3. Variety — Multiple data types from diverse sources: Structured (SQL tables), semi-structured (JSON/XML), unstructured (images, videos, audio).
4. Veracity — Quality, accuracy and trustworthiness of data. IoT sensor noise, duplicate records, and inconsistent social media posts challenge data quality.
5. Value — Actionable business insights extracted from processed data. Netflix's recommendation engine drives 80% of all viewing via data-derived suggestions.
[Diagram — Draw the Pentagon model with Big Data in center, V1–V5 at corners. Takes 60 seconds and earns 1–2 extra marks.]
[Conclusion]
Understanding the 5 Vs is critical for designing appropriate storage (Volume), streaming pipelines (Velocity), data lake architecture (Variety), data quality frameworks (Veracity), and analytics solutions (Value). Together, they define the full scope of Big Data challenges.
Keywords to Underline in Exam
| Topic | Underline These Keywords |
|---|---|
| 5 Vs | Volume, Velocity, Variety, Veracity, Value, fault-tolerant, scalable, distributed |
| HDFS | NameNode, DataNode, metadata, block replication, rack-aware, 128 MB, heartbeat, Secondary NameNode (checkpoint only) |
| MapReduce | Map phase, Shuffle & Sort, Reduce phase, key-value pairs, input splits, JobTracker, TaskTracker, Combiner |
| YARN | ResourceManager, NodeManager, ApplicationMaster, Container, multi-tenancy, Scheduler, resource allocation |
| Hive | HiveQL, Metastore, Driver, Compiler, Optimizer, Executor, schema-on-read, MapReduce translation |
| Spark | RDD, Resilient, Distributed, in-memory, transformation (lazy), action (eager), DAG, lineage, 100× faster |
| NoSQL | schema-less, horizontal scaling, BASE, eventual consistency, document, column-family, key-value, graph |
| HBase | HMaster, RegionServer, column-family, random access, HDFS-backed, row key, Region |
| Pig | Pig Latin, Grunt shell, ETL, LOAD, FILTER, FOREACH, STORE, translates to MapReduce |
Common Exam Mistakes — Avoid These
❌
Writing "Secondary NameNode is the backup of NameNode" — WRONG! Always deducted. It only checkpoints metadata (merges fsimage + edit logs). The HA NameNode (Hadoop 2.x) is the actual standby backup.
❌
Not drawing diagrams — AKTU examiners explicitly reward labelled diagrams. Even a rough box-and-arrow diagram for HDFS, MapReduce, or YARN earns 1–2 additional marks.
❌
Confusing Pig and Hive — Pig = ETL/data pipeline scripting (Pig Latin); Hive = SQL-like ad-hoc querying (HiveQL). Never interchange their descriptions.
❌
Calling MapReduce a database — it is a distributed processing framework. Data is stored in HDFS; MapReduce only processes it in batch.
❌
Writing only 3 or 4 Vs — always include all 5 Vs: Volume, Velocity, Variety, Veracity, Value. Missing any V costs marks.
✅
Always start 7/10-mark answers with a formal definition sentence. "MapReduce is a distributed data processing framework developed by Google and implemented in Apache Hadoop..."
✅
Use bullet points for listing components, advantages, and challenges — they look organised and typically score higher than dense paragraphs.
✅
End every 7/10-mark answer with a conclusion sentence. "Thus, HDFS provides fault-tolerant, scalable, distributed storage for petabyte-scale Big Data applications."
Examiner Expectations — Section-wise
Section A (2-mark answers)
• One clear definition sentence
• 2–3 crisp bullet points with technical terms
• No diagram required (unless architecture question)
• Max 3–4 lines total
• Use keywords confidently (underline them)
• 2–3 crisp bullet points with technical terms
• No diagram required (unless architecture question)
• Max 3–4 lines total
• Use keywords confidently (underline them)
Section B & C (7-mark answers)
• Formal introduction paragraph
• Labelled diagram (mandatory for architecture Qs)
• Component-wise explanation with bullets
• 1–2 real-world examples
• Conclusion sentence tying it together
• Labelled diagram (mandatory for architecture Qs)
• Component-wise explanation with bullets
• 1–2 real-world examples
• Conclusion sentence tying it together
3-Day Strategy
Final 3-Day Exam Preparation Plan
A structured hour-by-hour schedule optimised for maximum marks in the AKTU Big Data exam (BCS061 / KCS061).
🏆
Target Strategy: Do ALL Section A first (14 easy marks) → pick best 3 from Section B → choose strongest option in each Section C question. This strategy alone = 60–68 marks if executed correctly.
1
Day 1 — Foundation & Core Hadoop (Unit 1 + Unit 2)⏰ Morning — 2 hours
- Memorise all 5 Vs with one real-world example each. Use mnemonic: "Very Vast Varieties Verify Value"
- Write the Analysis vs Reporting comparison table from memory (6 rows)
- Learn the 4 types of analytics: Descriptive, Diagnostic, Predictive, Prescriptive with "What happened / Why / What will / What to do?" triggers
- List 8 Big Data platforms and 3 types of digital data with examples
⏰ Afternoon — 3 hours
- Memorise all 11 Hadoop ecosystem components with their roles — write the table from memory and check
- Understand and draw the MapReduce flow diagram from scratch: Input → Splits → Mappers → Shuffle & Sort → Reducers → Output
- Practice the complete Word Count example through all MapReduce phases with intermediate data shown
- Hadoop Streaming — 3-line definition + one example command
⏰ Evening — 1 hour
- Mock: Write 2-mark answers for — What is Hadoop? What is MapReduce? What is Hadoop Streaming? Data formats in Hadoop?
- Self-check: Write all 11 Hadoop ecosystem tools from memory. Check against the table.
2
Day 2 — Architecture Deep-Dive (Unit 3 + Unit 4)⏰ Morning — 3 hours
- Draw HDFS Architecture diagram from memory: Client, NameNode, Secondary NameNode (checkpoint!), 3 DataNodes with rack labels, arrows
- Memorise HDFS Read (5 steps) and HDFS Write (7 steps) in correct order — write them out
- Critical facts: Block = 128MB, Replication = 3, Secondary NN = NOT backup, heartbeat = 3 sec, dead after 10 min
- Cloud Hadoop: 3 platforms (AWS EMR, Azure HDInsight, GCP Dataproc) + advantages (5 pts) + challenges (5 pts) table
⏰ Afternoon — 2.5 hours
- Draw YARN Architecture diagram from memory: Client, ResourceManager, 2 Worker Nodes with NodeManagers + ApplicationMasters + Containers
- YARN: Why YARN? (multi-tenancy), ResourceManager sub-components, Fair vs Capacity Scheduler
- Spark RDDs: R-D-D full expansion, transformations (5 examples) vs actions (5 examples), why 100× faster (in-memory vs disk)
- NoSQL 4 types: Key-Value/Document/Column-Family/Graph with one example and use case each
- MongoDB CRUD: Write all 4 syntax examples from memory with realistic data
⏰ Evening — 1 hour
- Timed mock: Write complete 7-mark HDFS answer (14 min) — must include the labelled architecture diagram
- Self-check: Can you write YARN's 4 components + roles from memory?
3
Day 3 — Ecosystem Tools + Full Mock (Unit 5 + Revision)⏰ Morning — 2.5 hours
- Draw and explain the Hive Architecture data flow: HiveQL → Driver → Compiler ↔ Metastore → Optimizer → Executor → HDFS → Result (7 boxes, 7 steps)
- Master the Pig vs Hive vs MapReduce vs SQL 9-row comparison table — reproduce from memory
- HBase: HMaster + RegionServer roles + HBase vs RDBMS comparison (7 rows)
- Pig Latin: Grunt shell, LOAD/FILTER/FOREACH/GROUP/JOIN/STORE — meaning and one example each
⏰ Afternoon — 2 hours
- Full mock: Attempt 7 Section A short answers (2-marks each, 3 min each = 21 min)
- Full mock: Write one complete Section C answer (7 marks, 14 minutes) — choose 5 Vs or HDFS architecture
- Review all 5 key diagrams: Pentagon (5 Vs), MapReduce Flow, HDFS Architecture, YARN Architecture, Hive Data Flow
- Decide in advance which 3 Section B topics you'll answer
⏰ Evening — 45 min
- Final read-through of keywords to underline (from the Keywords table in Model Answers section)
- Review the common mistakes list — commit to not making any of them
- Plan exam time: Section A = 25 min → Section C = 65 min → Section B = 55 min → Review = 15 min
⚡ One-Night Emergency Plan (Only 4 Hours Available!)
| Hour | Topics to Cover | Why — Guaranteed Marks |
|---|---|---|
| Hour 1 | 5 Vs of Big Data + All 11 Hadoop Ecosystem Components with roles | Section A + C guaranteed — 20–30 marks potential from these two topics alone |
| Hour 2 | MapReduce architecture diagram + HDFS architecture diagram with all labels | Sure-shot 7-mark questions — appear in every single AKTU Big Data paper without exception |
| Hour 3 | YARN architecture (4 components) + Spark RDDs + Hive architecture data flow | Very high probability Section B and Section C choices — covers 3 different units |
| Hour 4 | Pig vs Hive vs MapReduce comparison table + NoSQL 4 types + all 2-mark definitions | Section B options + complete coverage of all Section A short notes |
Exam Day Time Management
| Section | Time | Approach |
|---|---|---|
| Section A — 7 × 2 = 14 marks | 25 min | Attempt ALL 7. One definition sentence + 2 bullets each. Start here — builds confidence and guarantees 14 marks. |
| Section C — 5 × 7 = 35 marks | 65 min | ~13 min per question. ALWAYS draw a labelled diagram. Choose the option you know better for each pair. |
| Section B — 3 × 7 = 21 marks | 55 min | ~18 min each. Pick your 3 strongest topics. Never attempt unfamiliar topics under time pressure. |
| Review | 15 min | Check all diagrams are labelled. Verify correct number of questions attempted. Add any missing conclusions. |
🏆
Maximum Marks Strategy:
• Draw a labelled diagram for every architecture question — examiners award marks even when explanation is weak
• Underline key terms (NameNode, MapReduce, RDD, HiveQL) — signals technical accuracy to examiner
• Write a conclusion sentence for every 7-mark answer — "Thus, HDFS provides..." shows complete understanding
• Attempt Section A first to secure easy 14 marks before tackling harder questions
• In Section B, answer your best topic first — saves time and ensures your strongest answer is complete
• Draw a labelled diagram for every architecture question — examiners award marks even when explanation is weak
• Underline key terms (NameNode, MapReduce, RDD, HiveQL) — signals technical accuracy to examiner
• Write a conclusion sentence for every 7-mark answer — "Thus, HDFS provides..." shows complete understanding
• Attempt Section A first to secure easy 14 marks before tackling harder questions
• In Section B, answer your best topic first — saves time and ensures your strongest answer is complete
📌 5 Key Diagrams — What to Draw in Each Unit
Fig 1 — 5 Vs Pentagon (Unit 1)
- Pentagon shape with "Big Data" in center circle
- 5 coloured circles at pentagon corners: V1–V5
- Label each with name + one-word meaning
- Dashed lines from center to each corner
- ⏱️ Takes ~60 seconds to draw
Fig 2 — MapReduce Flow (Unit 2)
- Input → 3 Splits → 3 Mappers → Shuffle & Sort box → 2 Reducers → Output
- Label all boxes and all arrows with step numbers ①–⑥
- Show key-value (k,v) on arrows between phases
- ⏱️ Takes ~90 seconds to draw
Fig 3 — HDFS Architecture (Unit 3)
- Client → NameNode (metadata, left) → 3 DataNodes (data, bottom)
- Secondary NameNode connected to NameNode (checkpoint arrow)
- Label racks: Rack 1 (DataNode 1+2), Rack 2 (DataNode 3)
- ⏱️ Takes ~90 seconds to draw
Fig 4 — YARN Architecture (Unit 4)
- Client → ResourceManager (master) → 2-3 Worker Nodes
- Inside each worker: NodeManager + ApplicationMaster + Containers
- Label all arrows: "submit job", "allocate containers"
- ⏱️ Takes ~60 seconds to draw
Fig 5 — Hive Data Flow (Unit 5)
- Linear: User → Driver → Compiler → Optimizer → Executor → HDFS → Result
- Metastore connected to Compiler from below with dashed arrow
- Number each box ①–⑦ along the top
- ⏱️ Takes ~60 seconds to draw