Nelder-Mead Method with Local Selection Using Neighborhood and Memory for Stochastic Optimization

Abstract

We consider the Nelder-Mead (NM) simplex algorithm for optimization of discrete-event stochastic simulation models. We propose new modifications of NM to reduce computational time and to improve quality of the estimated optimal solutions. Our means include utilizing past information of already seen solutions, expanding search space to their neighborhood and using adaptive sample sizes. We compare performance of these extensions on six test functions with 3 levels of random variations. We find that using past information leads to reduction of computational efforts by up to 20%. The adaptive modifications need more resources than the non-adaptive counterparts for up to 70% but give better-quality solutions. We recommend the adaptive algorithms with using memory with or without neighborhood structure.