The "Mystery Scabbard" is a pommel holster for two revolvers (one on each side of the saddle). The flap is a little large, but effective for keeping the black powder revolver clean and dry.

Even though one of his contemporaries complained that he sat in the saddle "like a sack of meal." General Horace Porter said of him,

General Grant was a great rider, simply splendid. He could ride 40 or 50 miles and come in perfectly fresh and tire out younger men. He was much attached to a little horse named Jeff Davis because he was secured on Jeff Davis's plantation [near Vicksburg]. General Grant was the only man I ever saw, except one, who could go through a battle without flinching. He never lacked in courage, never dodged. He wouldn't as much wink when bullets went whizzing by. He had iron nerves. He was never hurt by a bullet, despite his exposure.

The blurred background indicates the the photograph was shot with a wide open lens to yield the shortest possible exposure, indicating the photographer was probably also worried about motion blur.

The well behaved (and gated) pony, originally "appropriated" from the farm of the brother of Jefferson Davis, was purchased by Grant for his personal use, and remained with him for many years after the war:
http://www.granthomepage.com/grantequestrian.htm

Any idea what the scabbard on the front of the saddle is supposed to hold? Wrong shape for a telescope or horse pistol, and I doubt the General needed an entrenching tool....

I know little about photography but if this photo was taken in 1865, how long would that horse have had to stand still to get such a clear photo? I see very little blur.

[Several seconds. - Dave]