You certainly have time to visit Bern. Will you be jet lagged?
My "quick trip to Berne" involves walking down from the Bahnhof to the Nydegg Bridge through the old town. Two places to eat by the bridge-
The Old Tram depot is a good place for the average traveler. Wide range of foods and beers.
The Brasserie Bärengraben ( across the street) has a sophisticated menu and very high quality.. Not cheap. Not very big. Hard to get in unless you are there early or between normal meal times.
From the Nydegg Bridge I always walk up the short steep climb to the Rose Garden. Splendid views of Bern, and a cafe as well.
To return, I sometime walk, or else catch the tram that stops in front of the Brasserie ( #12, I recall) and ride back up a few stops to the Bahnhof. Buy your ticket from the machine before you get on the bus. I think you need 1 or 2 zones ( see the map on the ticket machine) - tickets are priced by zones.
Regarding construction on the train line, on May 26th, when there is construction on any route, and especially a busy route like that one, a replacement bus is always provided if the train is completely blocked.
Biggest nuisance is hauling luggage on and off.
May 26th in particular - I have read the notice for May 26th, and note that the timetable already allows for the delay. Looks like it only adds a minute or so, but the number of trains may be reduced. I'd not worry about it; just use the timetable. If you miss one, the next one will be along in a short while.
Also, there is an alternate (local) line that goes by way of Belp (where the airport is) You could always take that. It is a pretty valley.
I am sure that visiting the caves would be interesting. My own approach to visiting Switzerland is to try do things that I cannot do elsewhere. So, I have never visited the caves during many trips to the region. On the other hand, I've seen quite few caves elsewhere....
<""4.The house I am renting has 2 bikes for us to use That is why I am
thinking of riding to the falls and Murren. Can anyone suggest a good
route that is mostly downhill?">
The entire floor of the Lauterbrunnen Valley from Lauterbrunnen to Stechelberg and beyond is almost flat, and offers lots of things to stop and stare at.
Down hill rides could be quite steep.. You can take a bike
I'd not bother to go up to Mürren...my opinion.