How to Configure your Domain Filters and Website Blacklists

This section will enable you to configure your domain filters and website blacklists. You will be able to add a list of keywords that a website url must either contain or not contain. Simply speaking, we are looking to include or exclude our set of keywords from domain names. If you are unsure about this section, you can always leave it as is. The domain filter works exceptionally well with niches that have a set of global keywords that define the niche. For example, cryptocurrency, CBD and Hemp, Vape and so on. Let's take the CBD niche. By default, most websites will have the keywords CBD and Hemp in them because these keywords define the actual niche. Even so, some website domains may not have these keywords. A good example being botanicals, canna, etc. Of course, the domain filter will help you to scrape a more relevant set of results but by default, it will restrict your results to only those websites that contain your keywords in the url. Likewise, you can choose to exclude a list of websites from scraping. Such websites could include large marketplaces such as Ebay, Amazon, Alibaba. As such websites have a lot of products, you are bound to come across them and so, to save time, it is sensible to exclude such sites. Likewise, you could exclude popular newspapers and magazines if you are looking for business leads.

Domain filters can help to improve the quality of your results only if you know what you are doing and you know your niche inside out. Remember, not all websites will have your keywords in the url as there are a lot of branded sites out there.