BRADLEY UNIVERSITY

ELECTRICAL & COMPUTER ENGINEERING

Dr. Gary L. Dempsey

• University Position: Associate Professor of ECE
• Campus Phone: (309) 677-2801
• Campus Office: Jobst Hall, # 339

EE535-Engineering Applications of Neural Networks

• EE 535 Lecture Workbook (700 pages), G. Dempsey, January 2007
• References:
  1. Understanding Neural Networks and Fuzzy Logic, Stamatios Kartalopoulos, IEEE Press, 1996.
  2. Neural Networks: A Comprehensive Foundation, Simon Haykin, Macmillan Publishing.
  3. MATLAB Neural Network Toolbox, The Math Works Inc.
  4. Parallel Distributed Processing, Rumelhart, McClelland, & PDP Group, The MIT Press.
  5. Neurocomputing, Robert Hecht-Nielsen, Addison-Wesley Publishing.
  6. Neural Networks for Control, Miller, Sutton, and Werbos, eds., The MIT Press.

• Prerequisites:
  EE302 or graduate standing, knowledge of MATLAB, and a high-level language (Basic, C, etc.)
• Course Description:

The course will provide the student with a working knowledge of the theory, design and applications of artificial neural networks. Emphasis will be directed to low-level implementations such as embedded microcontrollers and integrated circuits. Specific architectures such as correlation matrix memory, perceptron, adaline, multilayer networks, radial-basis function networks, and Hopfield networks will be examined as well as their corresponding learning rules (least mean square, back-propagation, etc.). Image, signal, and speech processing, system identification, control systems and optimization applications will be discussed. The engineering simulation program MATLAB and the Neural Network Toolbox will be incorporated into the lectures, homework, and tests.
• Course and Grading Policy:

Regular attendance is expected. Prior consent must be given before postponement of exam. Unexcused absence for examinations will receive a grade of zero. The percentage of a missed exam due to illness will be added to the final exam percentage (no makeup exams). Late homework assignments will not be accepted for grading without prior consent. Homework problems will not count as part of course grade (2004 change) but are expected to be completed. Solutions are not provided but work is graded thoroughly. Your grade in EE 535 is determined from your performance on three exams and a final comprehensive exam. Borderline grades are determined by homework, class participation and attendance.

Homework
Three Exams 69%
Final Exam 31%


• Off-Campus Students:

The videotaped ANN class will only be offered at the start of the Spring semester. The course completion date can be no later than the last Friday in June. Off-campus students normally complete the course approximately 2 weeks after the normal semester. A grade of IP (in-progress) will be given at the end of the semester and a contract will be sent to the student which will require a signature stating the course will be completed by the last Friday in June. If the course is not completed by the deadline a permanent grade of IC (incomplete) will be assigned. The course requires MATLAB and the Neural Network Toolbox.
• Preliminary Outline:

1. Neural network introduction, major researchers, applications. Lectures 1-3.
2. Hardware neurons, neural network architectures. Lecture 4.
3. Perceptron, classification, training examples, decision boundaries, linear algebra review, Matlab review. Lectures 4-6.
4. Linear associative memory, Hebbian learning, correlation matrix memory, associative memory applications. Lectures 7-9.
5. Speech recognition application. Lecture 10.
6. Neural Network Toolbox introduction. Lecture 11.
7. Homework discussion, worked examples. Lecture 12.
8. LMS algorithm, Adaline neuron, linear algebra review, classification application, Matlab use. Lectures 13,14.
9. Adaline simulations, error surface curves, Matlab use. Lectures 15-17.
10. Back-propagation algorithm, multilayer networks, XOR problem, local minima discussion. Lectures 18,19.
11. Back-propagation algorithm limitations, overfitting, underfitting, learning rate and weight initialization methods, momentum learning, adaptive learning rate method. Lectures 20-22.
12. Function approximation applications, character recognition application, image and data compression applications. Lectures 23,24.
13. HW #15 discussion. Lecture 25.
14. Temporal neural architectures, nonlinear prediction, adaptive noise cancellation, and nonlinear filtering applications. Lectures 26-28.
15. System identification applications, control theory review. Lecture 29.
16. Nonlinear controller design. Lecture 30.
17. Neural control versus conventional control, CMAC neural network. Lecture 31, 32.
18. Neural network hardware implementation issues, Phase-locked loop application. Lecture 33.
19. Radial-basis function networks. Lecture 34.
20. Hopfield networks, CAM applications, optimization network, neural A/D converter application. Lecture 35.
21. Introduction to fuzzy networks, HW #16 discussion, final exam. Lecture 36.

• Homework Assignments:

Homework 1,2: Neural network theory, definitions, short answer, true/false.
Homework 3-6: Perceptron training for classification application.
Homework 7,8: Speech recognition application.
Homework 9,10: Associative memories.
Homework 11,12: Adaline training for classification application.
Homework 13: Back-propagation algorithm- XOR problem.
Homework 14: Mapping application and character recognition application.
Homework 15: Speech recognition application and nonlinear filter application.
Homework 16: Noise cancellation application.
Homework 17: Controls Application: CMAC Neural Network.

• Introduction to Neural Networks:

Best Introduction Site Selected by 2003 Class
Leonard Clark's Application Site
Namik Akyil and Adam Raferty Application Site