IT603(B) • Data Mining • Unit IV

Classification, Prediction and Cluster Analysis

Complete RGPV exam-oriented notes on supervised learning, decision trees, Bayesian classification, neural networks, support vector machines, evaluation methods, k-means, hierarchical clustering, DBSCAN and outlier detection.

Start Unit 4 Notes
Home › Data Mining › Unit 4

1. Classification 14 Marks

Classification is a supervised technique that learns from labeled data and assigns new records to predefined classes.

Components

  • Training data
  • Class labels
  • Classification model
  • Test data
  • Predicted class
Labeled Training Data | Learning Algorithm | Classification Model | New Record | Predicted Class

Applications

  • Spam detection
  • Disease diagnosis
  • Credit approval
  • Customer churn prediction

2. Prediction 14 Marks

Prediction estimates an unknown or future numerical value from existing data.

Examples

  • Sales forecasting
  • House-price prediction
  • Demand estimation
  • Student marks prediction
Y = a + bX
ClassificationPrediction
Predicts categorical labelsPredicts continuous values
Example: Pass/FailExample: Marks

3. Classification and Prediction Process 14 Marks

  1. Collect and preprocess data.
  2. Split data into training and testing sets.
  3. Train a model.
  4. Evaluate its performance.
  5. Select and deploy the best model.
Data → Preprocessing → Training → Evaluation → Deployment

4. Decision Tree Induction 14 Marks

A decision tree contains attribute-test nodes, outcome branches and class-label leaves.
Attendance? / High Low | | Assignment? Fail / Good Poor | | Pass Fail

Steps

  1. Select the best splitting attribute.
  2. Partition records.
  3. Repeat recursively.
  4. Stop when a leaf condition is reached.
  5. Prune weak branches.

Advantages

  • Easy to understand
  • Handles numerical and categorical data
  • Produces readable rules

5. Attribute Selection Measures 14 Marks

Entropy

Entropy(S) = − Σ pᵢ log₂(pᵢ)

Information Gain

Gain(S,A) = Entropy(S) − Weighted Entropy after Split

Gain Ratio

Gain Ratio = Information Gain / Split Information

Gini Index

Gini(S) = 1 − Σ pᵢ²
MeasureUsed In
Information GainID3
Gain RatioC4.5
Gini IndexCART

6. Bayesian Classification 14 Marks

Bayesian classification predicts classes using probability and Bayes' theorem.
P(C|X) = [P(X|C) × P(C)] / P(X)

Naive Bayes

It assumes that attributes are conditionally independent given the class.

P(Cᵢ|X) ∝ P(Cᵢ) × Π P(xₖ|Cᵢ)

Advantages

  • Fast
  • Works well for text data
  • Needs less training data

7. Neural Network Classification 14 Marks

A neural network contains interconnected artificial neurons that learn complex relationships.
Input Layer → Hidden Layer → Output Layer
Output = f(Σ wᵢxᵢ + b)

Backpropagation

  1. Initialize weights.
  2. Perform forward pass.
  3. Calculate error.
  4. Backpropagate error.
  5. Update weights.

8. Support Vector Machine 14 Marks

SVM finds a hyperplane that maximizes the margin between classes.

Terms

  • Hyperplane
  • Margin
  • Support vectors
  • Kernel function
Class A: ● ● ● ---------------- Maximum-Margin Hyperplane Class B: × × ×

Kernels

  • Linear
  • Polynomial
  • RBF
  • Sigmoid

9. Model Evaluation 14 Marks

Methods

  • Holdout
  • Random subsampling
  • k-fold cross-validation
  • Stratified cross-validation
  • Bootstrap

Confusion Matrix

Predicted PositivePredicted Negative
Actual PositiveTPFN
Actual NegativeFPTN
Accuracy = (TP + TN) / Total
Precision = TP / (TP + FP)
Recall = TP / (TP + FN)
F1 = 2PR / (P + R)

10. Cluster Analysis 14 Marks

Clustering groups similar unlabeled objects together and separates dissimilar objects.

Requirements

  • High intra-cluster similarity
  • Low inter-cluster similarity

Methods

  • Partitioning
  • Hierarchical
  • Density-based
  • Grid-based
  • Model-based

11. K-Means Algorithm 14 Marks

K-means divides objects into k clusters by minimizing distance from cluster centroids.
  1. Select k.
  2. Initialize k centroids.
  3. Assign each point to the nearest centroid.
  4. Recalculate centroids.
  5. Repeat until stable.
SSE = Σ Σ ||x − μᵢ||²

Limitations

  • Requires k in advance
  • Sensitive to outliers
  • Sensitive to initial centroids

12. Hierarchical Clustering 14 Marks

Hierarchical clustering creates nested clusters represented by a dendrogram.

Agglomerative

Starts with individual objects and repeatedly merges the nearest clusters.

Divisive

Starts with one large cluster and repeatedly divides it.

Distance | | +--- A | +---+ | | +--- B +----+ | +--- C +---+ +--- D

Linkage

  • Single
  • Complete
  • Average
  • Centroid

13. DBSCAN 14 Marks

DBSCAN forms clusters from dense regions and marks low-density points as noise.

Parameters

  • Eps: neighbourhood radius
  • MinPts: minimum points required

Point Types

  • Core point
  • Border point
  • Noise point

Advantages

  • Finds arbitrary-shaped clusters
  • Detects noise
  • Does not need k

14. Outlier Analysis 14 Marks

An outlier is an object that differs significantly from normal data behaviour.

Types

  • Global outlier
  • Contextual outlier
  • Collective outlier

Methods

  • Statistical
  • Distance-based
  • Density-based
  • Clustering-based
An outlier may be an error, fraud, fault or important rare event.

Important Comparisons

ClassificationClustering
SupervisedUnsupervised
Uses labeled dataUses unlabeled data
K-MeansDBSCAN
Requires kUses Eps and MinPts
Weak noise handlingGood noise handling
Spherical clustersArbitrary shapes

Unit 4 Quick Revision

  • Classification predicts categorical classes.
  • Prediction estimates continuous values.
  • Decision trees use attribute tests and leaves.
  • Naive Bayes applies Bayes' theorem.
  • SVM finds a maximum-margin hyperplane.
  • Cross-validation evaluates generalization.
  • K-means uses centroids.
  • Hierarchical clustering creates a dendrogram.
  • DBSCAN detects density-based clusters and noise.

Important RGPV Exam Questions

Long Answer Questions

  1. Explain classification and prediction.
  2. Explain decision-tree induction with diagram.
  3. Explain information gain, gain ratio and Gini index.
  4. Explain Bayesian classification.
  5. Explain neural-network classification.
  6. Explain support vector machine.
  7. Explain classifier-evaluation methods.
  8. Explain k-means with example.
  9. Explain hierarchical clustering.
  10. Explain DBSCAN and outlier analysis.

Short Answer Questions

  1. Define entropy.
  2. What is pruning?
  3. State Bayes' theorem.
  4. What is a support vector?
  5. Define centroid.
  6. What is a dendrogram?
  7. What is a core point?

Download Study Resources

Unit 4 PDF

Printable notes will be uploaded soon.

Coming Soon

Solved Algorithms

Decision tree and k-means examples coming soon.

Coming Soon

PYQ Analysis

Repeated questions coming soon.

Coming Soon

Frequently Asked Questions

Classification labeled data par predefined class predict karta hai, jabki clustering unlabeled data mein natural groups discover karta hai.
Pruning weak branches remove karke overfitting kam karti hai.
DBSCAN arbitrary-shaped clusters aur noise points detect kar sakta hai.