An indefinite nonlinear diffusion problem in population gentics,I: Existence and limiting profiles

We study the following Neumann problem Math eqution presented whereδ is the Laplace operator, ω is a bounded smooth domain in Rⁿ with νas its unit outward normal on the boundary∂ω and g changes sign in ω. This equation models the "complete dominance" case in population genetics of two alleles. We show that the diffusion rate d and the integral ∫_ωg dx play important roles for the existence of stable nontrivial solutions, and the sign of g(x) determines the limiting profile of solutions as d tends to 0. In particular, a conjecture of Nagylaki and Lou has been largely resolved. Our results and methods cover a much wider class of nonlinearities than u²(1 - u), and similar results have been obtained for Dirichlet and Robin boundary value problems as well.