Complex systems have certain common properties, with power law statistics being nearly ubiquitous. Despite this commonality, we show that a variety of mechanisms can be responsible for complexity, illustrated by the example of a lattice on a Cayley Tree. Because of this, analysis must probe more deeply than merely looking for power laws, instead details of the dynamics must be examined. We show how optimality—a frequently-overlooked source of complexity—can produce typical features such as power laws, and describe inherent trade-offs in optimal systems, such as performance vs. robustness to rare disturbances. When applied to biological systems such as the nervous system, optimality is particularly appropriate because so many systems have identifiable purpose. We show that the "grid cells" in rats are extremely efficient in storing position information. Assuming the system to be optimal allows us to describe the number and organization of grid cells. By analyzing systems from an optimal perspective provides insights that permit description of features that would otherwise be difficult to observe. As well, careful analysis of complex systems requires diligent avoidance of assumptions that are unnecessary or unsupported. Attributing unwarranted meaning to ambiguous features, or assuming the existence of a priori constraints may quickly lead to faulty results. By eschewing unwarranted and unnecessary assumptions about the distribution of neural activity and instead carefully integrating information from EEG and fMRI, we are able to dramatically improve the quality of source-localization. Thus maintaining a watchful eye towards principles of optimality, while avoiding unnecessary statistical assumptions is an effective theoretical approach to neuroscience.