Pretty sure it's a reference for the SS, there's even a poster behind the employer with an image resembling that of the nazi's organization.
The swastika is a commonly used religious image in japan, having a long history in Buddhism.
The poster is a shogun clan crest I believe, these often involve a thick outer circle with the main crest inside. Interestingly there used to actually be a crest which was a swastika (or manji) inside. The Hachisuka. It predated the Nazi party a couple of centuries please mind.
I believe a few of these are still in use today, most famously being the Mitsubishi logo, which is a combination of two crests.
The the fact that the one on the wall is western characters in a traditional crest and is probably supposed to signify the shadyness of the organisation. As for the SS connection... I doubt it. Japan doesn't really teach much about that era even regarding Imperial Japans horrible crimes, let alone the Nazi ones. If the Japanese latch onto them it's usually because they think the uniforms are cool. That and the organisation doesn't seem to have much parallels. Though I'm on my first chapter so maybe they will change.
Now I am by no means an expert on japanese heraldry so I would recommend doing your own research into the subject as I'm sure it's a fascinating rabbit hole.