This paper examines the problem of designing a hybrid photovoltaic (PV)/battery system to achieve maximum power point tracking (MPPT) 'by design', or, passively, without requiring active control. In contrast to the existing literature on passive PV MPPT, a key goal of the paper is to derive analytic design rules for achieving passive MPPT, as well as dynamic models for the potential departure from MPPT over time. We use a 'self-balancing', hybrid PV array and Lithium (Li) ion cell integration topology to demonstrate the idea of 'MPPT by design' in this work. A small signal analysis is performed on the system to understand the effects of changing solar irradiation on the state of the system and its self-balancing behavior. In this analysis, we further identify design parameters of the system that enable the hybrid system to achieve MPPT. Finally, a simulation study validates the theoretical propositions of this paper.