The easiest way to get to Penza is from Moscow. A regular train (#052) departs from Kazanskiy Railway Station at about 8:30PM daily. It is quite comfortable and is affordable even for low-budget backpackers. In about 12 hours you will arrive at Penza's main railway station, "Penza I", at the centre of the city.
There is usually one slower (around 15 h) but cheaper overnight train to Penza, but cheap tickets to this train are often sold out well in advance. During periods of high demand (summer and around New Year) there may be another train, daily or several times a week. Also, it is possible to take one of many trains which stop in Penza then continue to other cities, but many of those trains are very dirty (especially ones heading to Central Asian countries. which may be less secure as well and are highly not recommended) and poorly heated during cold season.
There is a small airport in Penza, and, as of 2011, a flight from Moscow Domodedovo airport available several times a week, which will take around a hour. However, tickets are extremely expensive at 9750 руб for a return trip.
Penza does not have metro or trams, but the size and layout of the city make it easy to get around by bus (fare is 18 Russian rubles, about $0.30) or taxi (fare $2-5 depending on how far you need to go and how hard you bargain with the driver). Another option is to use shuttle bus service: it is slightly more expensive but faster and more comfortable compared to a regular bus.