ADmantX Service Fetcher Description

Why the ADmantX Semantic Analysis Service is Accessing Your Web Site Pages

The ADmantX Semantic Analysis Service provides real-time analysis of the context of the article or text on a web page. The ADmantX SA Service utilizes Natural Language Processing (NLP) technology to analyze text the way a human does when reading. The output of the ADmantX SA Service is the “context” (automatic categorization) of the web page. The ADmantX SA Service then provides the “context” output to the requesting client (our customers and partners).

Our customers are web site publishers who use our Semantic Analysis Service to provide “contextual ad targeting” as well as provide brand safety for advertisers (blocking ads on inappropriate or questionable content).

Our partners are DSPs, SSPs, Ad Exchanges and Ad Networks who utilize our contextual data for their businesses.

The Benefits of the ADmantX SA Service Analyzing Your Site’s Pages

If your company is a web site publisher who is a customer of ADmantX, the “context” data of your site’s pages is used in ad campaigns to drive the “right ad” to the “right page” at the “right time.” You will realize increased campaign performance because the ads placed are more closely relevant to what the page is about. Also, if there are pages that have content that is controversial or inappropriate for certain audiences the resulting context provided can identify those and provide a signal for an ad campaign to avoid placement on said page. The contextual signals provide a means for the protection of the brands that are buying your inventory.

If your company is not a web site publisher customer of ADmantX, then your web site’s inventory is being offered by one or more of our partners through RTB programmatic ad platforms. Our partners pass the URLs of your inventory to our ADmantX SA Service, which returns the “context” data about the page. The “context” is used by the requesting platform to select the relevant ads for your pages. Again, the more relevant the ad to the page the more likely a visitor will click the ad. Your advertisers will realize more traffic and you will realize more ad revenue. Brand safety is also obtained as ads can be blocked from pages with controversial or inappropriate topics. Your advertisers can be confident their brands’ reputation is safe.

How Does the ADmantX SA Service Access Your Site and How Often?

The ADmantX SA Service is not a web crawler or spider. It does not visit a site and drill down the site to all of the pages. It does visit individual web pages upon request via a web service call by our customers or partners. The web service request is generated as a result of a user visiting the page and it contains ad slots to be filled. Typically the ADmantX SA Service will access a page only once a day. In some cases there will be several daily accesses as geographically separate ADmantX SA Service environments can access your site globally. If you see more than several accesses to a specific page by the ADmantX SA Service please contact us at support@admantx.com and we will immediately handle your issue to resolution.

How Do You Identify Your Site Is Being Accessed by the ADmantX SA Service?

The ADmantX SA Service uses our “Fetcher” component to retrieve the HTML of the page requested via an HTTP or HTTPS request. The user-agent header will be in the following format:

admantx-<env>/<ver> (+https://www.admantx.com/service-fetcher.html)


<env> is an abbreviation for the service environment performing the access

<ver> is the version number of the ADmantX SA Service

The URL in the comment field is the address of this page.

What Happens to the Content of Your Web Pages That Are Analyzed?

The ADmantX SA Service extracts the static content of the page for the semantic analysis. No JavaScript is executed. The raw text of the page is used to produce our service’s contextual data, which is returned to the requestor.

Does the ADmantX SA Service Use Cookies for Its Operation?

No. The ADmantX SA Service does not use cookies at all.

How Often Will the ADmantX SA Service Access any Page on Your Site?

Each ADmantX SA Service will access a page on your site on demand via a web service request by one of our customers or partners. The number of times a page will be accessed is dependent on several factors:

  1. The success or the failure of the fetch of the page.
  2. If the page is being requested to different ADmantX SA Service environments.

Typically the fetch of a page on your site should occur no more than several times a day. If you are seeing many more requests for the same page by our user-agent then please contact us right away at support@admantx.com and we will immediately work on this to a resolution.

Does the ADmantX SA Service Recognize robots.txt?

The ADmantX SA is not a crawler/spider that drills down from the home page to all pages on the site. The ADmantX SA Service fetches pages on demand as a result of a user visit and uses cached results to avoid multiple accesses in a day. The requests to the ADmantX SA Service are generated by user visits and these are unlikely to access forbidden resources that are defined in a site’s robots.txt file. For these reasons the ADmantX SA Service does not refer to a site’s robots.txt file. If you are seeing accesses to site resources that are disallowed then please contact us right away at support@admantx.com and we will immediately work on this to a resolution.

ADmantX offers a solution that provides “Domain Categorization” and that is different from the page-level analysis provided by the ADmantX SA Service. In the ADmantX Domain Categorization Solution the robots.txt file of the site being analyzed is read and honored.