Neural Network Computing and Natural Language Processing
DOI:
https://doi.org/10.1558/cj.v5i4.63-75Keywords:
neural networks, parallel distributed processing, associative learning, natural language processingAbstract
After twenty years of disfavor, a technology has returned which imitates the processes of the brain. Natural language experiments (Sejnowski & Rosenberg: 1986) demonstrate that neural network computing architecture can learn from actual spoken language, observe rules of pronunciation, and reproduce sounds from the patterns derived by its own processes. The consequences of neural network computing for natural language processing activities, including second language acquisition and representation, machine translation, and knowledge processing may be more convulsively revolutionary than anything imagined in current technology. This paper introduces neural network concepts to a traditional natural language processing audience.References
Brown, C. "Parallel Optics: Solution to Von Neumann Bottleneck?" Electronic Engineering Times (24 November 1986), 35.
Brown, C. "Parallel Processors Bypass Von Neumann Bottleneck," Electronic Engineering Times (21 April 1986), 58.
Business Week (26 January 1987): "They're here: Computers that 'think.'"
Cohen, M., S. Grossberg, and D.G. Stork, "Recent Developments in a Neural Model of Real-Time Speech Analysis and Synthesis," IEEE First International Conference on Neural Networks, San Diego, 21-24 June 1987.
Cowan, J.D., and D.H. Sharp, "Neural Nets," Preprint, 1987.
Goldstine, Herman H. The Computer from Pascal to von Neumann. Princeton: Princeton University Press, 1972.
Hebb, D.O. The Organization of Behavior. New York: Wiley, 1949.
Hinton, Geoffrey E., and Terrence Sejnowski. "Optimal Perceptual Inference," Proceedings of the IEEE Computer Science Conference on Computer Vision and Pattern Recognition. Washington, DC (1983), 448-453.
Hopfield, John J. "Neural networks and physical systems with emergent collective computational abilities," Proceedings of the National Academy of Sciences, 79 (1982) 2554.
Hopfield, John J. and David W. Tank, "Computing with Neural Circuit: A Model," Science, 233 (8 August 1986), 625-633.
Huang, N.-K. "A Learning Experiment on English Spelling Rules," IEEE First International Conference on Neural Networks, San Diego, 21-24 June 1987.
Insight, 4, 7 (15 February 1988), 8-12: "Making Machines in Mind's Image."
Johnson, R.C. "Neural Networks Naive, Says Minsky," Electronic Engineering Times (3 August 1987), pp. 41, 52.
Johnson, R.C. "Nestor Frees Neural Nets from Hardware Chains," Electronic Engineering Times (1 February 1988), pp. 33-34.
Klimasauskas, C.C., ed. The 1987 Annotated Neuro-Computing Bibliography. Sewickley, PA: NeuroConnection, 1987.
McCorduck, Pamela. Machines Who Think. San Francisco: W.H. Freeman, 1979.
McCulloch, W.S., and W. Pitts. "A logical calculus of the ideas immanent in nervous activity," Bull. Math. Biophys. 5 (1943), 115.
Minsky, Marvin, and S.A. Papert. Perceptrons: an Introduction to Computational Geometry. Cambridge, MA: MIT Press, 1969.
Nature, 323 [1986], 533.
Neumann, J. von. "Probabilistic logics and the synthesis of reliable organisms from unreliable components," Automata Studies, C.E. Shannon and J. McCarty, eds. Princeton: Princeton University Press, 1956.
Newsweek, 110, 3 (20 July 1987) 52-53: "Mimicking the Human Mind."
Pierce, W.H. Failure Tolerant Computer Design. New York: Academic Press, 1965.
Pitts, W., and W.S. McCulloch. "How we know universals: the perception of auditory and visual forms," Bull, Math. Biophysics. 9 (1947), 127.
Rosenblatt, Frank. "The Perceptron: a probabilistic model for information storage and organization in the brain," Psychological Review, 62 (1958), 386.
Rumelhart, David, and James L. McClelland. "On Learning the Past Tense of English Verbs," Parallel Distributed Processing: Psychological and Biological Models, vol. 2. Cambridge, MA: MIT Press, 1986.
Rumelhart, David, Geoffrey Hinton, and R.J. Williams, "Learning Internal representations by error propagation," Parallel Distributed Processing: Explorations in the microstructure of cognition. vol. 1. Cambridge, MA: MIT Press, 1986.
Schwartz, T.J. "Neural Net software for PC AT," Electronic Engineering Times (2 March 1987), pp. 27, 49.
Sejnowski, Terrence J., and Charles R. Rosenberg. "Parallel Networks that Learn to Pronounce English Text," Complex Systems, 1 (1987), 145-168.
Sejnowski, Terrence J., and Charles R. Rosenberg. "NETtalk: A Parallel Network that Learns to Read Aloud," JHU/EECS-86/01, Johns Hopkins University, EE/CS Department, January 1986.
Sejnowski, Terrence. "Speech and Signal Processing Tutorial," IEEE First International Conference on Neural Networks, San Diego, 21-24 June 1987.
Time, 131, 8 (22 February 1988), 53: "Supercomputers: The Fastest Brain in Town."
Waltz, Davis L., and Jordan B. Pollack, "Massively Parallel parsing: A Strongly Interactive Model of Natural Language Interpretation," Cognitive Science, 9 (1985), 51-74.
Widrow, B., and M.E. Hoff, "Adaptive switching circuits," WESCON [Institute of Radio Engineers, Western Electronic Show and Convention] Convention Record, 4 (1960), 96.
Willshaw, D.J., and Christian von der Malsburg, "How patterned neural connections can be set up by self-organization," Proceedings of the Royal Society, London B. 194 (1976), 431.
Winograd, S., and J.D. Cowan. Reliable Computation in the Presence of Noise. Cambridge, MA: MIT Press, 1963.
