Pattern Recognition Methods and Introduction to Machine Learning

Course description sheet

Basic information

Field of study: AGH UST International Courses
Major: All
Organisational unit: AGH University Database of Electives
Study level: University database of electives
Form of study: Full-time studies
Profile: General academic

Didactic cycle: 2024/2025
Course code: UBPOJOS.A200000.04820.24
Lecture languages: English
Mandatoriness: Obligatory
Block: General Modules
Course related to scientific research: No
USOS code: 240-INT-xS-125

Course coordinator

Witold Dzwinel

Lecturer

Witold Dzwinel

Period

Summer semester

Method of verification of the learning outcomes

Exam

Activities and hours

Lectures: 14
Laboratory classes: 14

Number of ECTS credits

Goals

C1	To familiarize students with the role of machine learning in modern technology and its role in artificial intelligence
C2	To make students acquainted with machine learning terminology.
C3	Familiarize them with basic ML tasks: feature engineering, classification, regression, clustering algorithms and validation methods
C4	To make students acquainted with the principles of developing simple ML systems.

Course's learning outcomes

Code	Outcomes in terms of	Methods of verification
Knowledge – Student knows and understands:
W1	Student knows and understands methods of data normalization and calibration	Activity during classes, Participation in a discussion, Execution of a project, Execution of laboratory classes, Examination, Oral answer
W2	Students have the knowledge allowing them to choose a proper data representation and use feature engineering to avoid "curse of dimensionality" problem	Activity during classes, Participation in a discussion, Examination, Oral answer
W3	Students have an extended expertise in classification, regression or clustering algorithms, validation and data visualization methods	Activity during classes, Execution of laboratory classes, Examination, Engineering project, Completion of laboratory classes
Skills – Student can:
U1	Students can develop an ML data analysis system based on classification, regression or clustering and validate the results.	Engineering project, Completion of laboratory classes
U2	The ability of using ML Python libraries.	Execution of laboratory classes, Engineering project, Completion of laboratory classes
Social competences – Student is ready to:
K1	The improvement of skills concerning public speaking.	Participation in a discussion
K2	The development of public relations through discussions with opponents.	Activity during classes, Participation in a discussion
K3	Student can present his/her results to the audience.	Activity during classes, Participation in a discussion

Program content ensuring the achievement of the learning outcomes prescribed to the module

Students will be provided with basics in machine learning and data analysis. They will understand the methodological construction of ML systems and to start more advanced courses.

Student workload

Activity form	Average amount of hours* needed to complete each activity form
Lectures	14
Laboratory classes	14
Preparation for classes	15
Realization of independently performed tasks	15
Examination or final test/colloquium	5
Contact hours	28
Preparation of project, presentation, essay, report	15

Student workload	Hours 106
Workload involving teacher	Hours 28

* hour means 45 minutes

Program content

No.	Program content	Course's learning outcomes	Activities
1.	Introduction 1. Artificial Intelligence methods in the modern world. AI taxonomy. 2. Who is a data scientist? 3. What is Machine Learning and why it is crucial for data analysis. 4. Five general principles data scientists should know Benfords Law, Law of Large Numbers, Zip's law, Bonferroni's Principle, Averages definitions. 5. ML in a nutshell: feature engineering, regression - classification - clustering, validation.	W1, U2, K2, K3	Lectures, Laboratory classes
2.	Data representation 1. Feature vector and feature space. 2. Features' normalization. 3. The properties of the high-dimensional spaces. 4. Curse of dimensionality. Strange properties of N-D spaces. 5. Feature engineering and data embedding --> how to diminish the dimensionality (basics) 6. Non-structured data. Sophisticated embedding.	W2, U2, K2	Lectures, Laboratory classes
3.	Data/Model representation 1. ML tasks: Classification - Regression and Clustering 2. Number of linear dichotomies. Underfit. 3. Vapnik-Chervonienkis dimension. 4. Distances and proximity measures. 5. Vector-Vector, Vector- Set, Set-Set, Pdf - pdf: Minkowsky distances, Mahalanobis, Proximity measures, Kullback-Leibler divergence, Wasserstein metrics, optimal transport theory. 6. Manifold learning irt distance - feature engineering.	W2, U2, K2	Lectures, Laboratory classes
4.	Model representation 1. Classification/regression - performance measure - loss functions 2. Optimizers (gradient descent, stochastic gradient descent, minibatch SGD, momentum, Nesterov, Adam, Adagrad, Adadelta etc.) 3. Training, validation, testing: terminology, one hit, cross-validation, ROC, AUC, Precision/Recall, F1.	W1, W2, U2, K2, K3	Lectures, Laboratory classes
5.	Feature engineering 1. Sparse coding hipothesis. 2. Feature selection algorithms (filters, wrappers) 3. Feature extraction methods: Principal component analysis (PCA), Linear Discriminant Analysis (LDA), Fischer algorithm 4. Autoencoders. 5. Random Projection and Johnson-Lindenstruss theorem. 6. Non-linear methods: Multidimensional scaling (MDS), Isomap, t-SNE.	W1, W2, U2, K2, K3	Lectures, Laboratory classes
6.	Simple classifiers 1. Minimum distance (maximum discriminant) classifiers 2. Bayesian rule and clussifiers - the best classifier. 3. Naive Bayes and k-NN classifier. 4. Kernel k-NN and the methods of finding nearest neighbors (kd-tree, Fridmann algorithm)	W2, U1, U2, K2, K3	Lectures, Laboratory classes
7.	Advanced classifiers 1. Linear classifiers (preceptron, Fischer, liner SVM) 2. Support Vector Machine classifier (regression). 3. Ensemble classifiers. 4. Neural Networks (MLP, DNNs, Hopfield, RBM and DBNs) 5. Bagging and Boosting. 6. Imbalanced classes.	W3, U1, U2, K1, K3	Lectures, Laboratory classes
8.	Clustering 1. Expectation maximization algorithm (EM). Generative models (DBN) 2. Clustering algorithms - taxonomy. 3. Hierarchical algorithms (Ward, Jarvic-Patrick, mutual nearest neighborhood others) 4. Partitioning clustering: k-Means, kernel k-Means, fuzzy-k-Means 5. Modern clustering schemes: DBSCAN, Chameleon, SNN, CURE, Affinity propagation. 6. Clustering validation: purity, entropy, Silhouette plots.	W2, W3, U1, U2, K1, K2	Lectures, Laboratory classes
9.	Pattern recognition - basics 1. Data acquisition and preparation. Machine Learning vs Pattern Recognition 2. Convolution principles, filters 3. Main transforms: Fourier, wavelets, Karhunen-Loeve, Hough 4. Introduction to Convolutional Neural Networks	W1, W3, U1, U2, K2	Lectures

Extended information/Additional elements

Teaching methods and techniques :

Lectures, Discussion, E-learning

Activities	Methods of verification	Credit conditions
Lectures	Activity during classes, Participation in a discussion, Execution of a project, Execution of laboratory classes, Examination, Oral answer
Lab. classes	Activity during classes, Participation in a discussion, Execution of laboratory classes, Engineering project, Completion of laboratory classes

Method of determining the final grade

The final grade is calculated as the weighted sum of the grades from oral (or test) exam and the lab activities. The weights are the same and equal to 0.5. The studens with excellent (5.0) grade from lab excercises are exempt from oral exam.

Prerequisites and additional requirements

The passed courses in Calculus, Algebra and Numerical Methods. Good knowledge of Python.

Rules of participation in given classes, indicating whether student presence at the lecture is obligatory

Lectures: Students participate in the classes by learning the subsequent contents according to the course syllabus. Students should systematically ask questions and clarify doubts. Audiovisual recording of the lecture requires the consent of the instructor. Laboratory classes: Students perform laboratory exercises following the materials provided by the instructor. The student is required to prepare in the subject of the exercise, which can be verified by a colloquium in oral or written form. Credit for classes is given based on a presentation of solutions to problems posed. The pass mark for the module is given after the successful completion of all laboratory classes.

Literature

Obligatory

Bishop, C. M. (2006). Machine learning and pattern recognition. Information Science and Statistics. Springer, Heidelberg.
Grossman R., L., Karnath, Ch, Kegelmeyer, P., Kumar, V., Namburu, R.,R., Data Mining for Scientific and Engineering Applications, Kluwer Academic Publisher, 2001
R.Duda, P.Hart, D. Stork, Patterns Classification, Wiley Interscience, 2001.

Optional

Theodoris S and Koutroumbas K, Pattern Recognition, Academic Press, San Diego, London, Boston, 1998.
Mitra, S. and Acharya T., Data Mining: Multimedia, Soft Computing and Bioinformatics, 424pp. J. Wiley, 2003.