That type of problem is usually caused by the batteries or battery charger. Since the cart is running for a little while after the batteries are recharged it sounds like the batteries to me.
If the charger is acting normally - with a green light when plugged into the wall that turns red when plugged into the cart and wall at the same time - and then turns green when the batteries are recharged, then the charger is operating normally which points towards the batteries needing to be replaced.
Sara