You're definitely not alone and you definitely do not need to have read the books or to have played the games to Get Things - most of the people I know have been absorbing lore through fic anyway. You might have to google a few characters that are going to appear in the next season (usually, the other witchers of the school of the wolf Lambert, Eskel and Vesemir) but luckily youtube is full of cutscenes from the games and the wikia is pretty well maintained, if you need info.

I'm so glad to see so many people interested in the witcher on dw!