Astonishing resemblance to "Palazzo Vecchio" in Florence, Italy.

[More specifically, the Palazzo Pubblico in Siena. -tterrace

On the lower left of the photo, you can see stairs coming down from 34th street. At this location, just out of view to the left is the southern portal of the Park Avenue tunnel, which ends at 39th. The tunnel, originally called the Murray Hill Tunnel, carried trains of the New York and Harlem RR (predecessor of the New York Central), but was converted to a streetcar line in the 1850's after NYC banned steam engines south of 42nd street. In the photo it is still a streetcar tunnel, but soon after it will be converted to a vehicular tunnel (I believe around the time the current Grand Central Terminal opened in 1910).

The subway entrance (33rd street station on the 6) is still in the same spot, albeit without the ornate stairway covering.

Here is an explanation of the conduit tracks you see in the street, courtesy of a New York City traction fan.

That's not the end of a line, but rather a connection from a carhouse. The near track is actually the uptown-bound track of the NY & Harlem's 4th & Madison Aves. conduit car line. It continues into the tunnel at 34th St, through the tunnel to 42nd St., then doglegs west on 42nd to Madison and the uptown. The downtown-bound track is hidden by the roadway ramp. The other track switching into the uptown track is the ladder track for the carbarn which then existed between 33rd and 32nd Sts. The non-electrified track curving into 33rd St. might possibly have led to a back entrance into the carbarn, but I'm not sure how it could have been used. It does not show in a 1907 track map.

The tracks that led into conduit carbarns were usually few in number, but had conduit. Generally, they led to a transfer table, about halfway back into the barn, and all the other tracks were accessed only via the transfer table. Some tracks used only for storage might have had conduit; any tracks with pits could not. The transfer table had conduit.

How did cars get moved? In the early days probably pushing them by hand - as was the usual practice for horse or cable cars. Going into the barn, a running start off of the conduit power on the transfer table would allow the car to roll to the end of the track.

While the old kiosk entrances to the subway were removed by the 1960s, you can still enter through one at Astor Place and 4th Avenue. Here is a photo of it from Google street view.

in 1926, what are those covered stairwells leading to?

[Subway access. -tterrace]

Hard to believe that this grand structure was demolished in the 1970s to make way for 3 Park Avenue. I wonder how many millions of bricks and man-hours went into the 1905 structure?

Here's the replacement for this Armory, a 41 Story Office Building known as 3 Park Avenue. Built in 1976, the first 11 floors house the failing NYC Norman Thomas High School. When the school-day ends, the 33rd Street exit looks like a war zone with a NYPD presence befitting a World Series Game at Yankee Stadium. There are efforts underway to eventually close it, the High School, not the High Rise.

Now this is a defensible fortress.