Optimizing neural network models: Motivation and case studies

Practical successes have been achieved with neural network models in a variety of domains, including energy-related industry. The large, complex design space presented by neural networks is only minimally explored in current practice. The satisfactory results that nevertheless have been obtained testify that neural networks are a robust modeling technology; at the same time, however, the lack of a systematic design approach implies that the best neural network models generally remain undiscovered for most applications. This paper first presents an experimental study that demonstrates the complex interdependencies between various parameters of neural models. We then present an approach, based on genetic algorithms, for designing optimized neural networks for specific applications. Two case studies are discussed in which the benefits of a systematic ctesign method are exemplified. These studies are on real data sets that are relevant to the power industry. The flexibility of genetic optimization also permits some novel twists on neural modeling: overparametrization, input selection, and the synthesis of network architectures well suited for problem classes can be directly addressed.