BRADLEY UNIVERSITY

DEPT. OF ELECTRICAL & COMPUTER ENGINEERING & TECHNOLOGY

Dr. Gary L. Dempsey

• University Position: Associate Professor of ECE
• Campus Phone: (309) 677-2801
• Campus Office: Jobst Hall, # 339
• E-mail address: gld@cegt201.bradley.edu

EE410/691-Special Topics: Artificial Neural Networks

• Textbook: none required
• References:
  1. Understanding Neural Networks and Fuzzy Logic, Stamatios Kartalopoulos, IEEE Press, 1996.
  2. Neural Networks: A Comprehensive Foundation, Simon Haykin, Macmillan Publishing.
  3. MATLAB Neural Network Toolbox, The Math Works Inc.
  4. Parallel Distributed Processing, Rumelhart, McClelland, & PDP Group, The MIT Press.
  5. Neurocomputing, Robert Hecht-Nielsen, Addison-Wesley Publishing.
  6. Neural Networks for Control, Miller, Sutton, and Werbos, eds., The MIT Press.

• Prerequisite:
  EE301, EE302, knowledge of MATLAB, and a high-level language (Basic, C, etc.)
• Course Description:

The course will provide the student with a working knowledge of the theory, design and applications of artificial neural networks. Specific architectures such as correlation matrix memory, perceptron, adaline, multilayer networks, radial-basis function networks, and Hopfield networks will be examined as well as their corresponding learning rules (least mean square, back-propagation, etc.). Image, signal, and speech processing, system identification, control systems and optimization applications will be discussed. The engineering simulation program MATLAB and the Neural Network Toolbox will be incorporated into the lectures, homework, and tests.
• Course and Grading Policy:

Regular attendance is expected. Prior consent must be given before postponement of exam. Unexcused absence for examinations will receive a grade of zero. Late homework assignments will not be accepted for grading without prior consent. Independent work is required for homework assignments. Your grade in EE 410/691 is determined from your performance on two exams and homework. Borderline grades are determined by class participation and attendance.

Homework 30%
Midterm 30%
Final Exam 40%


• Off-Campus Students:

The videotaped ANN class will only be offered at the start of the Spring semester. The course completion date can be no later than the last Friday in June. Off-campus students normally complete the course approximately 2 weeks after the normal semester. A grade of IP (in-progress) will be given at the end of the semester and a contract will be sent to the student which will require a signature stating the course will be completed by the last Friday in June ( June 30 this year). If the course is not completed by the deadline a permanent grade of IC (incomplete) will be assigned. The course requires MATLAB and the Neural Network Toolbox.
• Preliminary Outline:

1. Neural network introduction, major researchers, applications. Lectures 1,2.
2. Hardware neurons, neural network architectures. Lecture 3.
3. Perceptron, classification, training examples, decision boundaries, linear algebra review, Matlab review. Lectures 4-6.
4. Linear associative memory, Hebbian learning, correlation matrix memory. Lectures 7,8.
5. Associative memory applications. Lecture 9.
6. Speech recognition application. Lecture 10.
7. Neural Network Toolbox introduction. Lecture 11.
8. Homework discussion, worked examples. Lecture 12.
9. LMS algorithm, Adaline neuron, linear algebra review, classification application, Matlab use. Lectures 13,14.
10. Adaline simulations, error surface curves, Matlab use. Lectures 15-17.
11. Back-propagation algorithm, multilayer networks, XOR problem, local minima discussion. Lectures 18,19.
12. Back-propagation algorithm limitations, overfitting, underfitting, learning rate and weight initialization methods, momentum learning, adaptive learning rate method. Lectures 20-22.
13. Function approximation applications, character recognition application, image and data compression applications. Lectures 23,24.
14. Neural network hardware implementation issues. Lecture 25.
15. Temporal neural architectures, adaptive noise cancellation application, nonlinear filtering applications, Matlab use. Lectures 26-28.
16. Prediction applications, genetic algorithms. Lecture 29.
17. System identification applications, control theory review. Lecture 30.
18. Nonlinear system identification and controller design. Lecture 31.
19. Neural control versus conventional control. Lecture 32.
20. Radial-basis function networks. Lecture 33.
21. CMAC neural network for control systems. Lectures 34,35.
22. Hardware/software implementation of multi-layer networks- Phase-locked loop application. Lecture 36.
23. Hopfield networks, CAM applications. Lecture 37.
24. Hopfield networks, optimization network. Neural A/D converter application. Lecture 38.
25. Introduction to fuzzy networks, fuzzy logic, membership functions, fuzzification, defuzzification. Lecture 39.

• Homework Assignments for 2000):

Homework 1,2: Neural network theory, definitions, short answer, true/false.
Homework 3-6: Perceptron training for classification application.
Homework 7,8: Speech recognition application.
Homework 9,10: Associative memories.
Homework 11,12: Adaline training for classification application.
Homework 13: Back-propagation algorithm- XOR problem.
Homework 14: Mapping application and character recognition application.
Homework 15: Speech recognition application and nonlinear filter application.
Homework 16: Noise cancellation application.
Homework 17: To be decided by student. Optional for undergraduates.
In addition two reports are required: (1) 2 page report on 2 Webites: 1 site on introductory ANNs, 1 site on ANN applications. Report due Friday March 10. (2) Each student will analyze all sites and select top 3 in each category. 1-2 page analysis. Due Friday April 21. The top 3 sites will be used in the next ANN course.
• Introduction to Neural Networks:
Neural Networks at Pacific Northwest National Laboratory