If you want to use electric fencing you should get some up ASAP so they learn they can't challenge it. That is about half the battle with keeping smart animals in. Once they learn they CAN get out it is way harder to keep them in.
You might be able to set up, or borrow, a solar electric setup for training purposes. "Shore" power is generally preferable though, it is way easier to maintain because shorts to ground don't kill the battery!

Baby animals are so cute!!!

P.S. Don't let a baby goat do anything you don't want an adult goat to do. Like stand on you....