Brahms has given good advice.
This link will help you better understand all the options in the Bernese Oberland and the Jungfrau region:
Follow the link shown there for the Jungfrau region.
Also, look at:
www.myswissalps.com/ac tivities/scenictrips/t rain
Alpenrose recently posted a trip report with excellent photos of the Brienzer Rothorn trip:
Scroll down a couple of days in that thread.
This map will help you visualize the trip(s). Zoom in for rail lines. Turn on transport stops/stations under "Points of Interest/Traffic" in the menus.
The time table will help you plan those trips:
In particular, you might find time for some walks or hikes, depending on how much time you spend on the train trips. Please be aware that the higher alpine trails are only becoming available in mid-June. Winter is just finishing at altitudes above 2000 meters or so. Some of the cableways resume operation in early June after the between seasons break.
Weather will govern your choice of trips. On a cloudy day ( at high altitude) you will only see the inside of clouds from the Jungfrau. That is expensive and not so enjoyable. Keep in touch via weather reports, web-cams,. local train stations, tourist offices. With a two day stay in the region, there is a a significant chance that you will not have weather good for the Jungfrau trip. The links above can give you some other ideas as contingency plans.
Your interest in travel suggests Interlaken as a good base. For a better experience of alpine villages, a bit more remote from good main-line rail connections, I prefer Wengen, which is just opening up in early June from low season ( "Low season" means not much skiing, therefore not many customers in April and May). Or, Grindelwald, or Mürren, or even Lauterbrunnen ( on the valley floor.)
Where else will you be in Switzerland? If you are within travel distance, you might be able to get to the Jungfrau on a different (sunny) day than your original plans indicate.