The Conciergerie is the oldest surviving part of the original royal palace on the Île de la Cité, and unlike so much of Paris, it looks really medieval:
For a view from a similar angle, painted in 1412, see the June page from the Très Riches Heures of the Duc de Barry, a famous late-medieval book of hours.
It is also possible to see fragments of the city wall built in the twelfth century under the king Philippe Auguste, still surviving here and there around the city:
This guard tower protrudes into a schoolyard (or possibly a park, it was hard to tell) in the Marais, the historic (at least since the 15th century) Jewish quarter of Paris. The Marais is full of kosher delis and Judaica shops, and we'll be going back to visit more thoroughly; this time what I noticed, besides a lot of challah and chopped liver, was the memorial plaques on every school we passed, commemorating the children who had been deported from the school with the cooperation of the Vichy regime from about 1942-1944.