I'm going to explain something from an American perspective.
Over in Europe, alternative games like adventure games and stuff had already taken hold. Stuff like Broken Sword were immensely popular in Europe. So Europeans tend to typically be more open to different kinds of gaming styles.
That's really different here in America. In America, we are pretty much dead set on action or games that feature immediately apparent gratification.
Shenmue was different. It's a game that you take your time playing, it's a game that you let stew, and let it become a part of you. It's similar to games like Animal Crossing in that regard, in that you don't rush it. You take your time and enjoy it.
I think that, back then, gaming, at least in America, mostly couldn't understand or get the appeal of a game like Shenmue where you work a part time job, and can't fight people at any time. But I think things are different now and there's a lot of alternative games going, including the aforementioned Animal Crossing, that could make III a success. I also think there's something about different audiences. Nintendo fans for instance, are willing and more open to more experimental gaming styles than the typical gamer, and I think that if Shenmue I and II were on Gamecube, they could could have found success there rather than Xbox.
I feel putting Shenmue II on Xbox was a huge mistake.
I also think that despite the intention of being mass market, Shenmue I and II was far too ahead of its time to become that.
But I think with the right push and the right audience, it can find its market and audience now. Both Sony and PC gaming fans have a thing for alternative games. I think that if YSnet can gather the funds, and if it can run it, they should put III on Nintendo's NX as well.
So when discussing whether Shenmue is mass market or not, I think it really depends on the territory as well as potential audience. Xbox, I would say there, Shenmue is niche. On Nintendo platform, I'd say it's probably mass market. In America, there's POTENTIAL for mass market but it's not intrinsically mass market. In Europe, Shenmue is definitely capable of being mass market if it isn't that already. China, it's mass market for sure. Japan?
No, I don't think so. Definitely niche there. It's a loaded question, because different territories and different systems have different tastes.