We investigate a delay minimization problem in a multiple access wireless communication system. We consider a discrete-time non-fading additive white Gaussian noise (AWGN) multiple access channel. In each slot, bits arrive at the transmitters randomly according to some distribution, which is LLd. from user to user and from slot to slot. Each transmitter has an average power constraint of P. Our goal is to allocate rates to users, from the multiple access capacity region, based on their current queue lengths, in order to minimize the average delay of the system. We formulate the problem as a Markov decision problem (MDP) with an average cost criterion. We first show that the value function is increasing, symmetric and convex in the queue length vector. Taking advantage of these properties, we show that the optimal rate allocation policy is one which tries to equalize the queue lengths as much as possible in each slot, while working on the dominant face of the capacity region.