Re: Let me get this right.
"Exiting the browser", as in using the File -> Close menu item, generally doesn't do anything more than close the current window. A pop up/under window is usually another instance of the browser and therefore a different process which is unaffected by closing a different instance to it. Closing a window will close all the tabs in it - although Microsoft are doing their level best to break this standard as much as possible in IE/edge of course.
Yes, the symptom will be that you have no visible browser windows open however you may notice one in the OS's task bar. Some OSes, such as Windows Vista and 7, particularly in non-aero mode, make noticing whether or not an application is running or if it's just a launch icon very difficult. An application usually has to register a window with the OS's shell user interface in order to show as a switchable task, as a result it is relatively easy to hide a running task entirely - this does vary between OS shells though.