AKTU Big Data & Analytics Notes & Exam Prep

PYQ Frequency Analysis

Topic	Unit	Frequency	Priority	Section Pattern
5 Vs of Big Data	I	4/4	🔥 Must Do	Always in Sec A + C Q3
HDFS Architecture	III	4/4	🔥 Must Do	Sec B + Sec C Q5 every year
MapReduce Architecture	II	4/4	🔥 Must Do	Sec B Q2c every year
NoSQL vs RDBMS	IV	4/4	🔥 Must Do	Sec B Q2d, Sec C Q6
YARN Architecture	IV	3/4	Very Imp	Sec C Q4 always
Spark / RDD	IV	3/4	Very Imp	Sec C Q6b alternating
Apache Hive Architecture	V	3/4	Very Imp	Sec B Q2e, Sec C Q7
Apache Pig / Pig Latin	V	3/4	Very Imp	Sec C Q7, comparisons
Hadoop Ecosystem	II	3/4	Important	Sec B Q2b, Sec A short
Flume & Scoop	III	2/4	Important	BCD061 paper, Sec C Q4
HBase	V	2/4	Important	Sec C Q7a KCS061
Analysis vs Reporting	I	2/4	Important	Sec B + Sec C alternating
Cluster Setup / Specification	III	2/4	Moderate	BCS061 2025 Sec B
Master-Slave Replication	II	1/4	Moderate	BCS061 2025

Unit-wise Analysis & Sure-Shot Questions

Unit I Introduction to Big Data & Analytics

Easy High Weight ▼

🔥 5Vs Analysis vs Reporting Big Data Types Analytics Types Big Data Platform Security & Privacy

Core Concept (simple language)

Unit I introduces what Big Data is and why it matters. Big Data = data so large/fast/varied that traditional tools can't handle it. Think of data from Facebook, YouTube, IoT sensors — all generated every second. The key exam topics are the 5 Vs (Volume, Velocity, Variety, Veracity, Value) and distinguishing analysis from reporting.

Sure-shot questions — Unit I

🔥 Describe the 5 Vs of Big Data with real-world examples for each. Why are they essential for data management?

4/4 papers 7 marks Sec C Q3 MOST REPEATED

⭐ Distinguish between Data Analysis and Reporting in Big Data context. How does advanced analytics go beyond traditional reporting?

3/4 papers 7-10 marks Sec B / C

⭐ Explain the types of Big Data analytics: Descriptive, Diagnostic, Predictive, Prescriptive with examples.

2/4 papers 7 marks Sec B Q2a

📝 What are different types of digital data? Give examples of structured, semi-structured, and unstructured data.

2/4 papers 2-5 marks Sec A short

✦ Examiner Expects in 5V Answer

Volume — TB/PB scale data, e.g., Facebook generates 4PB/day
Velocity — real-time streaming, e.g., Twitter 500M tweets/day
Variety — structured (SQL), semi (JSON/XML), unstructured (video)
Veracity — data quality/accuracy, noisy or uncertain data
Value — business insights derived, ROI from data analysis
Draw a pentagon/5V diagram — fetches 1-2 extra marks

Unit II Hadoop & MapReduce

Medium High Weight ▼

🔥 MapReduce Architecture 🔥 Hadoop Ecosystem MapReduce Application Dev Hadoop Streaming Master-Slave Replication

Core Concept

Hadoop is an open-source framework to store and process Big Data in a distributed cluster. MapReduce is Hadoop's processing engine — it breaks big tasks into smaller parallel tasks. Think of it like a restaurant: Map = each chef prepares their dish separately; Reduce = waiter combines and serves everything.

Sure-shot questions — Unit II

🔥 Explain the detailed architecture of MapReduce — Mapper, Reducer, Combiner, Partitioner, Shuffle & Sort phases with diagram.

4/4 papers 7-10 marks Diagram Required MUST PREPARE

🔥 Explain Apache Hadoop ecosystem components and how they work together for distributed storage and processing.

3/4 papers 7-10 marks Diagram Required

⭐ Discuss the steps involved in developing a MapReduce application — writing, testing (MR unit), and deploying.

2/4 papers 7-10 marks

📝 Discuss Master-Slave and Peer-to-Peer replication in Hadoop in detail.

1/4 papers 7 marks BCS061 2025

✦ MapReduce Diagram — Must Know Flow

Input Splits → each split sent to one Mapper
Mapper → processes, emits key-value pairs
Combiner → local aggregation (mini reducer)
Partitioner → decides which Reducer gets which keys
Shuffle & Sort → groups same keys together
Reducer → final aggregation → Output
Word Count example = most expected practical example

Unit III HDFS & Hadoop Environment

Medium Highest Weight ▼

🔥 HDFS Architecture 🔥 HDFS Read/Write Flume & Scoop Cluster Setup Block Sizes & Replication Cloud Deployment

Core Concept

HDFS = Hadoop Distributed File System. Stores huge files split into 128MB blocks across many machines. NameNode = brain (stores metadata, doesn't store data). DataNode = worker (stores actual data blocks). Each block replicated 3 times for fault tolerance. Think of it like a library: NameNode is the index catalogue, DataNodes are the book shelves.

Sure-shot questions — Unit III

🔥 Explain the design and architecture of HDFS — NameNode, DataNode, Secondary NameNode, block storage, replication factor with diagram.

4/4 papers 7-10 marks Diagram Required HIGHEST FREQUENCY

🔥 Explain how HDFS reads and writes files — step-by-step sequence of operations, pipeline writing.

3/4 papers 7-10 marks Flow Diagram

⭐ Describe the process of Data Injection using Flume and Scoop. How does the client interact with NameNode?

2/4 papers 7 marks BCD061 2025-26

📝 Discuss advantages and challenges of deploying Hadoop in cloud (AWS, Azure, GCP). How does cloud enhance scalability?

2/4 papers 7-10 marks

✦ HDFS Write Pipeline — Memorise This!

1. Client asks NameNode for block location
2. NameNode returns DataNode list (DN1, DN2, DN3)
3. Client writes to DN1 → DN1 pipes to DN2 → DN2 pipes to DN3
4. All 3 send ACK back through the pipeline
5. NameNode updates metadata after confirmation
Draw this pipeline with arrows — guaranteed marks

Unit IV YARN, NoSQL, MongoDB & Spark

Hard High Weight ▼

🔥 YARN Architecture 🔥 NoSQL vs RDBMS 🔥 Spark / RDD MongoDB CRUD Scala Basics Fair & Capacity Scheduler

Core Concept

YARN = Yet Another Resource Negotiator — Hadoop's resource manager. It separates resource management from processing. NoSQL = "Not Only SQL" — flexible databases for unstructured data (MongoDB, Cassandra, Redis). Spark = faster alternative to MapReduce because it processes data in memory (RAM) instead of disk. RDD = Resilient Distributed Dataset — Spark's core data structure.

Sure-shot questions — Unit IV

🔥 Explain the detailed architecture of YARN — ResourceManager, NodeManager, ApplicationMaster, Container with diagram.

3/4 papers 7 marks Diagram Required Always Sec C Q4

🔥 Define NoSQL databases. Compare NoSQL vs RDBMS — key characteristics, schema-less nature, use cases, advantages.

4/4 papers 7-10 marks Table comparison expected

🔥 Discuss Resilient Distributed Datasets (RDD) in Apache Spark — what are RDDs, transformations, actions, in-memory advantage.

3/4 papers 7-10 marks Sec C Q6b

⭐ Explain MongoDB CRUD operations with examples. How does MongoDB handle complex data structures vs traditional SQL?

3/4 papers 7-10 marks Sec C Q6

✦ NoSQL vs RDBMS — Quick Table (Must Include in Answer)

Schema: RDBMS = fixed schema | NoSQL = flexible/schema-less
Scalability: RDBMS = vertical (scale up) | NoSQL = horizontal (scale out)
ACID: RDBMS = full ACID | NoSQL = BASE (Basically Available)
Query: RDBMS = SQL | NoSQL = JSON queries / Key lookups
Best for: RDBMS = banking, ERP | NoSQL = social media, IoT, Big Data

Unit V Pig, Hive, HBase & Ecosystem Frameworks

Medium High Weight ▼

🔥 Pig vs Hive vs MapReduce 🔥 Hive Architecture HBase & Zookeeper Pig Latin Scripting HiveQL BigInsights / Big SQL

Core Concept

Apache Pig = high-level data flow language (Pig Latin). Procedural — you tell it HOW to process. Good for ETL (Extract-Transform-Load). Apache Hive = data warehouse, uses HiveQL (SQL-like). Declarative — you tell it WHAT you want. Good for analysis/reporting. HBase = NoSQL database built on HDFS, column-family store, real-time read/write.

Sure-shot questions — Unit V

🔥 Compare Apache Pig vs MapReduce vs SQL vs HIVE. Explain Pig Latin's role as a high-level scripting language.

3/4 papers 7 marks Table comparison Sec C Q7

🔥 Explain the architecture of Apache Hive — components, data flow, how HiveQL is translated to MapReduce jobs.

3/4 papers 7-10 marks Architecture Diagram

⭐ Explain HBase concepts — vs RDBMS, schema design, Zookeeper integration, advanced indexing.

2/4 papers 7-10 marks KCS061 Sec C Q7a

📝 Compare Local Mode vs MapReduce Mode in Apache Pig. When should a developer use Grunt shell vs a Pig script?

1/4 papers 7 marks BCD061 2025-26

✦ Pig vs Hive — The Table Examiners Love

Paradigm: Pig = Procedural (data flow) | Hive = Declarative (SQL-like)
Language: Pig = Pig Latin | Hive = HiveQL
Best for: Pig = ETL/data transformation | Hive = Data warehousing/queries
Schema: Pig = schema on use | Hive = schema on read
Users: Pig = programmers | Hive = data analysts (SQL background)

Most Important Diagrams (Text Form)

🔥 1. HDFS Architecture — Draw This Every Exam

┌─────────────────────────────────────────────────────────┐ │ CLIENT │ │ (reads/writes files via HDFS API) │ └───────────────────────┬─────────────────────────────────┘ │ 1. Asks for block location ▼ ┌─────────────────────────────────────────────────────────┐ │ NAMENODE (Master) │ │ • Stores metadata (file names, block map) │ │ • Does NOT store actual data │ │ • Single point of coordination │ │ ↔ Secondary NameNode (takes snapshots, checkpoints) │ └──────┬──────────────────────────────┬───────────────────┘ │ 2. Returns block locations │ ▼ ▼ ┌────────────┐ ┌────────────┐ ┌────────────┐ │ DataNode 1 │───▶│ DataNode 2 │───▶│ DataNode 3 │ │ Block A │ │ Block A │ │ Block A │ ← Replication Factor = 3 │ Block C │ │ Block B │ │ Block C │ └────────────┘ └────────────┘ └────────────┘ Worker Worker Worker (stores data) (stores data) (stores data) Block size = 128 MB (default) | Replication = 3 copies

🔥 2. MapReduce Architecture — Must Know

INPUT FILE (large) │ ▼ ┌───────────────────────────────────────────────┐ │ INPUT SPLITS (split file into chunks) │ │ Split-1 Split-2 Split-3 Split-4 │ └────┬──────────┬──────────┬──────────┬─────────┘ ▼ ▼ ▼ ▼ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ │MAP-1 │ │MAP-2 │ │MAP-3 │ │MAP-4 │ ← Parallel Mappers └──┬───┘ └──┬───┘ └──┬───┘ └──┬───┘ │ │ │ │ (key,value) pairs emitted by each Mapper │ ▼ ┌──────────────────────────────────────────┐ │ COMBINER (optional local reducer) │ │ Reduces data before network transfer │ └─────────────────────┬────────────────────┘ │ ▼ ┌──────────────────────────────────────────┐ │ SHUFFLE & SORT │ │ Groups same keys → sends to Reducers │ └────────┬─────────────────┬──────────────┘ ▼ ▼ ┌─────────┐ ┌─────────┐ │REDUCER-1│ │REDUCER-2│ ← Parallel Reducers └────┬────┘ └────┬────┘ │ │ ▼ ▼ OUTPUT-1 OUTPUT-2 → stored back to HDFS

🔥 3. YARN Architecture

┌──────────────────────────────────────────────────────┐ │ RESOURCE MANAGER (Master) │ │ ┌─────────────────┐ ┌──────────────────────────┐ │ │ │ Scheduler │ │ ApplicationsManager │ │ │ │ (allocates res) │ │ (manages app lifecycle) │ │ │ └─────────────────┘ └──────────────────────────┘ │ └────────────────────────┬─────────────────────────────┘ │ ┌──────────────┼──────────────┐ ▼ ▼ ▼ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ NODE MGR │ │ NODE MGR │ │ NODE MGR │ ← Worker Nodes │ ┌─────────┐ │ │ ┌─────────┐ │ │ ┌─────────┐ │ │ │Container│ │ │ │Container│ │ │ │Container│ │ │ │ (CPU+ │ │ │ │ (CPU+ │ │ │ │ (CPU+ │ │ │ │ RAM) │ │ │ │ RAM) │ │ │ │ RAM) │ │ │ └─────────┘ │ │ └─────────┘ │ │ └─────────┘ │ │ App │ │ App │ │ App │ │ Master │ │ Master │ │ Master │ └──────────────┘ └──────────────┘ └──────────────┘ Key: ResourceManager = boss | NodeManager = worker | ApplicationMaster = app coordinator | Container = execution unit

⭐ 4. Apache Hive Architecture

┌──────────────────────────────────────────────────┐ │ HIVE CLIENT INTERFACES │ │ CLI (Hive Shell) | JDBC/ODBC | Web UI │ └───────────────────┬──────────────────────────────┘ │ HiveQL Query ▼ ┌──────────────────────────────────────────────────┐ │ DRIVER │ │ Compiler → Optimizer → Executor │ │ (parses HiveQL → generates MapReduce plan) │ └────────┬──────────────────────────┬──────────────┘ │ │ ▼ ▼ ┌─────────────────┐ ┌─────────────────┐ │ METASTORE │ │ EXECUTION │ │ (schema/table │ │ ENGINE │ │ metadata) │ │ MapReduce │ │ MySQL/Derby │ │ / Tez / Spark │ └─────────────────┘ └────────┬────────┘ │ ▼ ┌─────────────────┐ │ HDFS │ │ (actual data) │ └─────────────────┘

AKTU Answer Writing Strategy

marks — Sec A

• 3–4 lines only
• Definition + 1 example
• No need for diagram
• Use bold keywords
• ≈ 40–60 words

marks — Sec C

• Introduction (2 lines)
• Main content (bullet + headings)
• Diagram (mandatory for arch Qs)
• Comparison table if asked
• Conclusion (1–2 lines)
• ≈ 250–350 words

marks — Sec B

• Proper introduction
• Labelled diagram
• Detailed point-wise content
• Real-world examples
• Advantages/disadvantages if applicable
• ≈ 400–500 words

✦ How to Start Architecture Answers

Start: "[Topic] is a [type] that [what it does]. It consists of the following components:"
Then draw the diagram
Then explain each component in 2–3 lines
End with: "Thus, [topic] enables [benefit] in Big Data environments."

✦ How to Write Comparison Answers

Always use a table with Feature | Option A | Option B columns
Minimum 5 comparison points
Add use cases at the end
Underline keywords in the table headers

✦ Diagram Tips for Max Marks

Use a ruler for boxes
Label every component
Use arrows to show data flow
Write a short caption below: "Fig: HDFS Architecture"
Diagrams alone can get 2–3 marks in a 7-mark question

Viva / Oral Questions

Q01What is the default block size in HDFS? Why is it large? — 128 MB. Large blocks reduce NameNode metadata overhead and increase throughput.

Q02What is the default replication factor in HDFS? — 3. Provides fault tolerance. Can be changed in hdfs-site.xml.

Q03What is the role of NameNode in HDFS? — Stores metadata (file names, block locations). Does NOT store actual data.

Q04What does YARN stand for? — Yet Another Resource Negotiator. Manages cluster resources in Hadoop 2.x.

Q05What is an RDD in Apache Spark? — Resilient Distributed Dataset. Immutable, distributed, fault-tolerant collection processed in parallel.

Q06How is Spark faster than MapReduce? — Spark stores intermediate results in RAM (in-memory), MapReduce writes to disk between stages.

Q07What is the difference between Pig and Hive? — Pig = procedural (Pig Latin), best for ETL. Hive = declarative (HiveQL/SQL-like), best for data warehousing.

Q08What is Zookeeper used for in HBase? — Provides distributed coordination — monitors cluster health, manages leader election.

Q09What are the 4 types of NoSQL databases? — Document store (MongoDB), Key-Value (Redis), Column-family (Cassandra), Graph (Neo4j).

Q10What is Flume used for in Hadoop? — Flume is a distributed service for collecting, aggregating, and moving streaming log data into HDFS.

Q11What is Sqoop? — Tool to transfer structured data between Hadoop and relational databases (MySQL, Oracle).

Q12What are the 5 Vs of Big Data? — Volume, Velocity, Variety, Veracity, Value.

Q13What does the Combiner do in MapReduce? — Performs local aggregation at the Mapper node to reduce data transferred over the network (mini-reducer).

Q14What is the ApplicationMaster in YARN? — An app-specific master that negotiates resources from ResourceManager and monitors task execution.

Q15What is the difference between transformation and action in Spark RDD? — Transformation = lazy (creates new RDD, e.g., map, filter). Action = triggers execution (e.g., count, collect).

Common Mistakes Students Make

❌

Writing only 4 Vs instead of 5 — Always include all 5: Volume, Velocity, Variety, Veracity, Value. Missing Veracity or Value loses marks.

❌

Confusing NameNode and DataNode roles — NameNode = metadata manager (NO data). DataNode = stores actual blocks. Mixing them is a major error.

❌

Skipping the Combiner in MapReduce diagrams — Many students draw only Mapper → Shuffle → Reducer. Always include Combiner as an optional step between Map and Shuffle.

❌

Not drawing labelled diagrams for architecture questions — For HDFS, MapReduce, YARN, Hive — a question without a diagram will lose 2-3 marks regardless of the text quality.

❌

Mixing up Pig and Hive — Pig = procedural, data flow, ETL. Hive = declarative, SQL-like, data warehousing. Get this wrong and your comparison table falls apart.

❌

Writing YARN as "Yet Another Resource Manager" — Correct full form: Yet Another Resource Negotiator.

❌

Saying MapReduce stores intermediate results in RAM — MapReduce writes to DISK between stages. SPARK stores in RAM. Getting this reversed will cost marks in comparison questions.

❌

Not adding examples for short answers — Even in 2-mark answers, a quick example (e.g., "MongoDB is a document store e.g., user profile JSONs") lifts you from average to above average.

3-Day Exam Strategy

Day 1

Foundation — High-frequency topics

5 Vs of Big Data + diagram
HDFS Architecture (full diagram)
HDFS Read + Write sequence
MapReduce Architecture (full diagram)
Prepare Word Count example for MapReduce
Short notes: Hadoop Streaming, Block sizes

Day 2

Ecosystem — YARN, NoSQL, Spark

YARN Architecture (full diagram)
NoSQL vs RDBMS comparison table
MongoDB CRUD operations
Spark RDD — transformations vs actions
Hive Architecture + data flow
Pig vs Hive vs SQL comparison

Day 3

Revision + Practice writing

Write full answers for Q1–Q5 from sure-shot list
Revise all 4 key diagrams without reference
Prepare HBase + Zookeeper short notes
Revise Flume & Scoop concepts
Read all viva questions (15 Qs)
Revise all keywords list

✦ Minimum Topics to Pass (Attempt safely)

5 Vs of Big Data

MapReduce Architecture

HDFS Architecture

YARN Architecture

NoSQL vs RDBMS

Pig vs Hive comparison

Important Keywords to Underline in Answers

Underline these words when they appear in your answers. Examiners scan for them.

Volume

Velocity

Variety

Veracity

Value

NameNode

DataNode

Block Size

Replication

Fault Tolerance

MapReduce

Mapper

Reducer

Combiner

Shuffle & Sort

ResourceManager

NodeManager

ApplicationMaster

Container

YARN

Schema-less

CAP Theorem

BASE

ACID

RDD

In-Memory

Lazy Evaluation

Transformation

Action

Pig Latin

HiveQL

Metastore

Zookeeper

Column Family

Horizontal Scaling

Distributed

Parallel Processing

Pipeline

Flume

Sqoop

One-Night Revision Topics

If you have only one night — study these and nothing else. These 8 topics alone can get you 50+ marks.

🔥 1. The 5 Vs

Volume · Velocity · Variety · Veracity · Value
One real example per V. Draw the pentagon.

🔥 2. HDFS Architecture

NameNode = metadata. DataNode = data blocks. Block = 128MB. Replication = 3. Draw the diagram with arrows.

🔥 3. MapReduce Flow

Input Splits → Map → Combiner → Shuffle+Sort → Reduce → Output. Draw the full flow. Word Count example ready.

🔥 4. YARN Architecture

ResourceManager + NodeManager + ApplicationMaster + Container. Draw the hierarchy. Know each component's role.

🔥 5. NoSQL vs RDBMS

5-row comparison table. Schema · Scaling · ACID · Query · Use case. 4 types of NoSQL: document, key-value, column, graph.

🔥 6. Spark RDD

RDD = immutable, distributed, fault-tolerant. Transformation (lazy) vs Action. Spark = in-memory. Spark > MapReduce in speed.

🔥 7. Pig vs Hive

Pig = procedural, Pig Latin, ETL. Hive = declarative, HiveQL, data warehouse. Draw Hive architecture. Quick comparison table.

🔥 8. HDFS Read/Write

Write: Client→NameNode→DN1→DN2→DN3→ACK. Read: Client→NameNode→nearest DataNode→stream data. Draw pipeline.

Big Data & Analytics
Complete Exam Guide

PYQ Frequency Analysis

Unit-wise Analysis & Sure-Shot Questions

Top 20 Sure-Shot Questions

Most Important Diagrams (Text Form)

AKTU Answer Writing Strategy

Viva / Oral Questions

Common Mistakes Students Make

3-Day Exam Strategy

Important Keywords to Underline in Answers

One-Night Revision Topics

Big Data & AnalyticsComplete Exam Guide

PYQ Frequency Analysis

Unit-wise Analysis & Sure-Shot Questions

Top 20 Sure-Shot Questions

Most Important Diagrams (Text Form)

AKTU Answer Writing Strategy

Viva / Oral Questions

Common Mistakes Students Make

3-Day Exam Strategy

Important Keywords to Underline in Answers

One-Night Revision Topics

Big Data & Analytics
Complete Exam Guide