This is basically what it does. Now whenever you move your mouse to a click-able link, CoolPreviews add-on will display a small icon which once clicked will give you a preview of that web link!. If you find it useful, then you don't even have to open a new tab/window anymore, since it uses the Firefox web rendering technology and emulates itself as a small web browser, you can read that web page as you would do with Firefox!.
But don't get fooled by the word "preview" since it won't give you instant page-loads or previews since it has to load the page from the scratch. CoolPreviews comes with 2 themes of its own and once opened, you can search for sites such as Google, Bing, Yahoo, BBC, Wikipedia, etc from its GUI as well.
Although it would be really annoying, but you can set it to automatically create previews every time you move your mouse over a URL link if you wish or assign a keyboard shortcut if you like (tempting :D).
Few other features to mention...
*. Zoom in or out (the whole page).
*. Zoom only the text.
*. View Google image results as a slideshow + shows itself next to images.
*. Integrates with Google SERP results (not actually necessary, since Google implemented the "preview" feature not that long ago).
*. Configure how fast the icon and the CoolPreview window will appear and close.
*. Enable/Disable the Add-on itself.
*. Remember the window position... are just among the main features to mention.
As said, it's an Add-on and uses the Firefox as the rendering engine thus it is as secure as the Firefox web browser as far as I'm concerned. If you want to install it then please visit this Firefox-Addon page.