If someone prefaces their question with "I'm not sure if this is a reference question, but . . . " it almost always turns out to be a completely appropriate question to ask a librarian. The opposite is true too. Someone who asks you to look up seven phone numbers and then calls back the next day to ask for them again, because they lost them, is not worried about the nature of their question.
The Encyclopedia Britannica's entry for library mentions catalogs, bibliographies, multimedia databases, indexes and abstracts as the librarian's tools. This is why the practice is called reference. But the questions I've been asked range from resembling a dog's breakfast (a great phrase I read in Hilary Mantel's Wolf Hall) to yes, requests to be pointed to written material on a narrow subject, or the name of a group offering a service.
So really there's no need to start your question with "I need a reference on . . " when all you want to know is where Harry Potter is. We won't send you away. We can also refer you to books on the knitting pattern for Dobby's socks, political issues raised in the HP novels, and the spirituality of Potterworld. But we might have to search the catalog for that.