Monday, August 27, 2012

MOSS Usage Reports explained


MOSS Usage Reports explained provides a very good insight into the kind the data for each parameter in the Site Collection Usage Summary page.

Mark Arend has written an excellent post with detailed descriptions of parameters displayed in MOSS 2007’s usage reports on the pages like SpUsageWeb.aspx (Site), SpUsageSite.aspx (Site Collection), SpUsageWebTopPages.aspx (Site), SpUsageSiteTopPages.aspx (Site Collection) and so on.

There are two report pages that are extremely useful, particularly for slightly smaller sites, that cannot be reached through the GUI interface in MOSS 2007. They are actually from the basic WSS system, and MOSS inexplicably misses out any direct reference though the administration pages.

  • -  /layouts/usage.aspx - This page brings data from the content database, which is the total hit for the page from ALL locations
  • -  /_layouts/usageDetails.aspx-  This URL brings data from ‘ SharedServices_DB ‘, which is processed thru multiple SQL table views and stored. It results only the hits to the page FROM A specific site collection.

In short, the data on the ‘usagedetails.aspx’ page is calculated for any hit (success or failure) to the location whereas the data in the ‘spUsageSite.aspx’ page shows the page which was accessed (and the number of times it was access in the Pie chart) FROM the site collection.

These are the definitions used by WSS and MOSS in summary usage reports (which are stored in the web metainfo):


  • Visit: A “total hit” that does not come from within the same server; that is, it either has no Referrer header, or it has a Referrer header from another server
  • Total hit: any hit that gets logged in the WSS http logs (we don’t log hits that result in error http results, or hits to the _layouts directory)
  • Hit: Anything in (2) except hits on files with these extensions: "gif","jpg","png","bmp","css","mid","wav","ico","xml","au","js","class"
  • Request:  Requests always measures Page Views, not all HTTP requests for individual items like images, style sheets, etc.

How Usage Analysis works

All WFEs behave in the same way as long as the Windows SharePoint Web Services service is running on each WFE. HTTP data from each WFE is collected and stored locally on disk. The method and process by which this data is persisted on disk is described in Usage Event Logging in Windows SharePoint Services 3.0. This behavior is the same for all WFEs.

How this data makes it back to DB, differs for WSS and MOSS


  • -  In WSS, a timer job called Usage Analysis, runs on each WFE and is responsible for parsing the usage log files and updating the information in the site’s content DB. 
  • -  In MOSS, a timer job called Office SharePoint Usage Analytics Log Import, runs on each WFE and is responsible for parsing the usage log files and uploading this data in the SSP DB (for SSP’s that have usage turned on, as per the instructions that you quote below)


  1. The Office SharePoint Usage Analytics Log Processing job is responsible for parsing and populating the usage report data in the SSP DB’s analytics tables (that use the ANL prefix)
  2. It runs every 15 min, to check is there’s new data imported (from the Office SharePoint Usage Analytics Log Import job) so that the reports are updated
  3. It also expires detailed data (kept only for 30 days) and report data (kept for 365 days)
  4. Windows SharePoint Services 3.0 generates usage event logs daily for each Web application when 'Enable logging' is selected on the Usage Analysis Processing page in SharePoint Central Administration
  5. When logging is enabled, Windows SharePoint Services by default creates log files in the 'C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\Logs' path on the front-end Web server, although you can specify an alternative location
  6. The Logs directory contains a folder for each Web application on the Web server, each named with a GUID that identifies the respective Web application
  7. Windows SharePoint Services 3.0 inserts an ampersand (&) between the top-level site URL and the sub site URL when it processes the log files
  8. This marks the log file as "processed" and prevents data from being counted twice if the usage processing job is accidentally run again on the same day

Usage Reporting in SharePoint (WSS or MOSS)


After usage reporting is enabled, site administrators and site collection administrators can view site usage summary pages that have the following information for their sites and site collections (SpUsageSite.aspx):
  • Requests and queries in the last day and the last 30 days.
  • Average number of requests per day over the last 30 days.
  • A chart of requests per day over the last 30 days.
  • A list of the top page requests over the last 30 days.
  • A list of top users over the last 30 days.
  • A chart of top referring hosts over the last 30 days.
  • A chart of top referring pages over the last 30 days.
  • A list of top destination pages over the last 30 days.
  • Top queries for the last 30 days (if search usage reporting is enabled).
  • Search results top destination pages (if search usage reporting is enabled).
SSP administrators for the search service can view a search usage reports page that tracks the following information (SpUsageSiteSearchQueries.aspx)
  • Number of queries per day over the previous 30 days.
  • Number of queries per month over the previous 12 months.
  • Top queries over the previous 30 days.
  • Top site collections originating queries over the previous 30 days.
  • Queries per search scope over the previous 30 days.
Site collection administrators for the SSP site can view a usage summary page that tracks the following information (usage.aspx)
  • Total amount of storage used by the site collection.
  • Percent of storage space used by Web Discussions.
  • Maximum storage space allowed.
  • Number of users for all sites in the hierarchy.
  • Total hits and recent bandwidth usage across all sites.
Enable Office SharePoint Usage Reporting

After Windows SharePoint Services usage logging is enabled in the server farm, SSP administrators must enable the Office SharePoint Usage Reporting service. SSP administrators can control the complexity of usage analysis processing, and select whether or not reporting is enabled for search queries. Please refer the following steps to enable portal usage reporting: 
  • On Central Administration home page, click the Shared Service Provider listed under Shared Services Administration in the Quick Launch bar.
  • On the SSP home page, in the Office SharePoint Usage Reporting section, click Usage reporting.
  • On the Configure Advanced Usage Analysis Processing page, in the Processing Settings section, click Enable advanced usage analysis processing.
  • In the Search Query Logging section, select Enable search query logging.
  • Click OK.
Note:  If advanced usage analysis processing is not selected, usage reporting statistics will be minimal.
Reset Internet Information Server 
  • Go to the Start button and click Run
  • Type IISReset and click OK.

Activate Office SharePoint Usage Reporting

After Office SharePoint Usage Reporting is enabled for the SSP, site collection administrators must activate the reporting feature. Until the reporting feature is activated on a site collection, usage reports are not available.

Please refer the following steps to activate the reporting feature: 
  • On the Site Actions menu, click Site Settings.
  • On the Site Settings page, in the Site Collection Administration section, click Site collection features.
  • On the Site Collection Features page, click the Activate button for the Reporting feature.

More Information

Whenever Usage Analysis is enabled, the Web Application Servers begin creating usage analysis logs in the ' ' path. There will be a separate folder named with a GUID that represents the web application. Within each of these folders will be a subfolder for each days logs which in turn contains usage logs in the format 01.log, 02.log etc. The usage analysis job runs against the data collected from the previous day(s) logs. For this reason running the Usage Analysis job more than once per day will not update usage data.

-  On a high level, this is how the usage reports are generated 

  • IIS keeps all the SharePoint usage records in memory and will only dump it into physical file (the usage log) when memory is full or IISRESET
  • By default, MOSS has daily timer job (Office SharePoint Usage Analytics Log Import ) to process these physical files into database's temp usage tables
  • Be default, MOSS has another hourly or minutely timer job (Office SharePoint Usage Analytics Processing) that process data in those temp/shadow tables into the real usage tables.
  • Once this is completed, usage report would then be available for viewing.
In MOSS, the two important timer jobs which are responsible for parsing, processing and updating the SSP DB (the ANL tables) are

‘Office SharePoint Usage Analytics Log Import’ and 'SharePoint Usage Analytics Log processing'.

These jobs run on each WFE in the farm on a daily basis to process the data in the shadow tables and write the final usage data into another set of tables like ANLHit.

‘Office SharePoint Usage Analytics Log Import’ job is responsible for parsing and populating the usage report data in the SSP DB’s analytics tables (that use the ANL prefix) and runs daily to pick up yesterday and only yesterday’s usage log files and parse them into the SSP table like 'ANLShadowHit' while 'SharePoint Usage Analytics Log Processing' job runs on hourly or minutely basis to process the data in the above shadow tables and write the final usage data into another set of tables like ANLHit.

The following categories are the most relevant for usage reports
  • Office SharePoint Usage Analytics Processing
  • Office SharePoint Usage Analytics Log Import
  • Microsoft.SharePoint.Administration.SPUsageAnalysisJobDefinition
  • Microsoft.SharePoint.Portal.Analytics.UsageProcessingJobDefinition
  • Microsoft.SharePoint.Portal.Analytics.LogImportJobDefinition

Report pages

  • View site collection reports   > http://sitecollection/_layouts/SpUsageSite.aspx
  • View site reports                  >  http://sitecollection/_layouts/SpUsageWeb.aspx 
  • Site Usage Report                >  http://sitecollection/_layouts/usageDetails.aspx 
  • Site Usage Summary            >  http://sitecollection//Usage.aspx
  • The http://_layouts/usageDetails.aspx even shows you the total hits for the documents which are very useful information.
Names of the application pages in the '_layouts' directory which show usage analysis data
  • usage.aspx
  • usageDetails.aspx
  • SpUsageSite.aspx
  • SpUsageWeb.aspx
  • SpUsageSiteQueries.aspx
  • SpUsageSiteResults.aspx