MOSS Usage Reports explained

MOSS Usage Reports explained provides a very good insight into the kind the data for each parameter in the Site Collection Usage Summary page.

Mark Arend has written an excellent post with detailed descriptions of parameters displayed in MOSS 2007’s usage reports on the pages like SpUsageWeb.aspx (Site), SpUsageSite.aspx (Site Collection), SpUsageWebTopPages.aspx (Site), SpUsageSiteTopPages.aspx (Site Collection) and so on.

There are two report pages that are extremely useful, particularly for slightly smaller sites, that cannot be reached through the GUI interface in MOSS 2007. They are actually from the basic WSS system, and MOSS inexplicably misses out any direct reference though the administration pages.

• -  /layouts/usage.aspx - This page brings data from the content database, which is the total hit for the page from ALL locations
• -  /_layouts/usageDetails.aspx-  This URL brings data from ‘ SharedServices_DB ‘, which is processed thru multiple SQL table views and stored. It results only the hits to the page FROM A specific site collection.

In short, the data on the ‘usagedetails.aspx’ page is calculated for any hit (success or failure) to the location whereas the data in the ‘spUsageSite.aspx’ page shows the page which was accessed (and the number of times it was access in the Pie chart) FROM the site collection.

These are the definitions used by WSS and MOSS in summary usage reports (which are stored in the web metainfo):

• Visit: A “total hit” that does not come from within the same server; that is, it either has no Referrer header, or it has a Referrer header from another server
• Total hit: any hit that gets logged in the WSS http logs (we don’t log hits that result in error http results, or hits to the _layouts directory)
• Hit: Anything in (2) except hits on files with these extensions: "gif","jpg","png","bmp","css","mid","wav","ico","xml","au","js","class"
• Request:  Requests always measures Page Views, not all HTTP requests for individual items like images, style sheets, etc.

How Usage Analysis works

All WFEs behave in the same way as long as the Windows SharePoint Web Services service is running on each WFE. HTTP data from each WFE is collected and stored locally on disk. The method and process by which this data is persisted on disk is described in Usage Event Logging in Windows SharePoint Services 3.0. This behavior is the same for all WFEs.

How this data makes it back to DB, differs for WSS and MOSS

• -  In WSS, a timer job called Usage Analysis, runs on each WFE and is responsible for parsing the usage log files and updating the information in the site’s content DB. 
• -  In MOSS, a timer job called Office SharePoint Usage Analytics Log Import, runs on each WFE and is responsible for parsing the usage log files and uploading this data in the SSP DB (for SSP’s that have usage turned on, as per the instructions that you quote below)

1. The Office SharePoint Usage Analytics Log Processing job is responsible for parsing and populating the usage report data in the SSP DB’s analytics tables (that use the ANL prefix)
2. It runs every 15 min, to check is there’s new data imported (from the Office SharePoint Usage Analytics Log Import job) so that the reports are updated
3. It also expires detailed data (kept only for 30 days) and report data (kept for 365 days)
4. Windows SharePoint Services 3.0 generates usage event logs daily for each Web application when 'Enable logging' is selected on the Usage Analysis Processing page in SharePoint Central Administration
5. When logging is enabled, Windows SharePoint Services by default creates log files in the 'C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\Logs' path on the front-end Web server, although you can specify an alternative location
6. The Logs directory contains a folder for each Web application on the Web server, each named with a GUID that identifies the respective Web application
7. Windows SharePoint Services 3.0 inserts an ampersand (&) between the top-level site URL and the sub site URL when it processes the log files
8. This marks the log file as "processed" and prevents data from being counted twice if the usage processing job is accidentally run again on the same day

Usage Reporting in SharePoint (WSS or MOSS)

After usage reporting is enabled, site administrators and site collection administrators can view site usage summary pages that have the following information for their sites and site collections (SpUsageSite.aspx):

• Requests and queries in the last day and the last 30 days.
• Average number of requests per day over the last 30 days.
• A chart of requests per day over the last 30 days.
• A list of the top page requests over the last 30 days.
• A list of top users over the last 30 days.
• A chart of top referring hosts over the last 30 days.
• A chart of top referring pages over the last 30 days.
• A list of top destination pages over the last 30 days.
• Top queries for the last 30 days (if search usage reporting is enabled).
• Search results top destination pages (if search usage reporting is enabled).

SSP administrators for the search service can view a search usage reports page that tracks the following information (SpUsageSiteSearchQueries.aspx)

• Number of queries per day over the previous 30 days.
• Number of queries per month over the previous 12 months.
• Top queries over the previous 30 days.
• Top site collections originating queries over the previous 30 days.
• Queries per search scope over the previous 30 days.

Site collection administrators for the SSP site can view a usage summary page that tracks the following information (usage.aspx)

• Total amount of storage used by the site collection.
• Percent of storage space used by Web Discussions.
• Maximum storage space allowed.
• Number of users for all sites in the hierarchy.
• Total hits and recent bandwidth usage across all sites.

Enable Office SharePoint Usage Reporting

After Windows SharePoint Services usage logging is enabled in the server farm, SSP administrators must enable the Office SharePoint Usage Reporting service. SSP administrators can control the complexity of usage analysis processing, and select whether or not reporting is enabled for search queries. Please refer the following steps to enable portal usage reporting: 

• On Central Administration home page, click the Shared Service Provider listed under Shared Services Administration in the Quick Launch bar.
• On the SSP home page, in the Office SharePoint Usage Reporting section, click Usage reporting.
• On the Configure Advanced Usage Analysis Processing page, in the Processing Settings section, click Enable advanced usage analysis processing.
• In the Search Query Logging section, select Enable search query logging.
• Click OK.

Note:  If advanced usage analysis processing is not selected, usage reporting statistics will be minimal.

Reset Internet Information Server 

• Go to the Start button and click Run
• Type IISReset and click OK.

Activate Office SharePoint Usage Reporting

After Office SharePoint Usage Reporting is enabled for the SSP, site collection administrators must activate the reporting feature. Until the reporting feature is activated on a site collection, usage reports are not available.

Please refer the following steps to activate the reporting feature: 

• On the Site Actions menu, click Site Settings.
• On the Site Settings page, in the Site Collection Administration section, click Site collection features.
• On the Site Collection Features page, click the Activate button for the Reporting feature.

More Information

Whenever Usage Analysis is enabled, the Web Application Servers begin creating usage analysis logs in the ' ' path. There will be a separate folder named with a GUID that represents the web application. Within each of these folders will be a subfolder for each days logs which in turn contains usage logs in the format 01.log, 02.log etc. The usage analysis job runs against the data collected from the previous day(s) logs. For this reason running the Usage Analysis job more than once per day will not update usage data.

-  On a high level, this is how the usage reports are generated 

• IIS keeps all the SharePoint usage records in memory and will only dump it into physical file (the usage log) when memory is full or IISRESET
• By default, MOSS has daily timer job (Office SharePoint Usage Analytics Log Import ) to process these physical files into database's temp usage tables
• Be default, MOSS has another hourly or minutely timer job (Office SharePoint Usage Analytics Processing) that process data in those temp/shadow tables into the real usage tables.
• Once this is completed, usage report would then be available for viewing.

In MOSS, the two important timer jobs which are responsible for parsing, processing and updating the SSP DB (the ANL tables) are

‘Office SharePoint Usage Analytics Log Import’ and 'SharePoint Usage Analytics Log processing'.

These jobs run on each WFE in the farm on a daily basis to process the data in the shadow tables and write the final usage data into another set of tables like ANLHit.

‘Office SharePoint Usage Analytics Log Import’ job is responsible for parsing and populating the usage report data in the SSP DB’s analytics tables (that use the ANL prefix) and runs daily to pick up yesterday and only yesterday’s usage log files and parse them into the SSP table like 'ANLShadowHit' while 'SharePoint Usage Analytics Log Processing' job runs on hourly or minutely basis to process the data in the above shadow tables and write the final usage data into another set of tables like ANLHit.

The following categories are the most relevant for usage reports

• Office SharePoint Usage Analytics Processing
• Office SharePoint Usage Analytics Log Import
• Microsoft.SharePoint.Administration.SPUsageAnalysisJobDefinition
• Microsoft.SharePoint.Portal.Analytics.UsageProcessingJobDefinition
• Microsoft.SharePoint.Portal.Analytics.LogImportJobDefinition

Report pages

• View site collection reports   > http://sitecollection/_layouts/SpUsageSite.aspx
• View site reports                  >  http://sitecollection/_layouts/SpUsageWeb.aspx 
• Site Usage Report                >  http://sitecollection/_layouts/usageDetails.aspx 
• Site Usage Summary            >  http://sitecollection//Usage.aspx
• The http://_layouts/usageDetails.aspx even shows you the total hits for the documents which are very useful information.

Names of the application pages in the '_layouts' directory which show usage analysis data

• usage.aspx
• usageDetails.aspx
• SpUsageSite.aspx
• SpUsageWeb.aspx
• SpUsageSiteQueries.aspx
• SpUsageSiteResults.aspx

Use a calculated column to group list items by month

=CONCATENATE("(",MONTH([Column_Name]),") - ",CHOOSE(MONTH([ Column_Name]),"January","February","March","April","May","June","July","August","September","October","November","December"))

Why is SSP removed from SharePoint 2010?

SSP is shared service provider and that was available in MOSS 2007. 

In MOSS, there are certain actions that can only be performed only if you have created the SSP. for instance: BDC and User profiles.

Now we all must be thinking that why SSP is removed from SharePoint 2010 version? 

Lets assume that you are going to have different web application and you would only need to work with Excel services and do not want to use any other service that comes under  Excel services , but still just to use one  Excel service you need to create two SSP that means two separate databases.

And in SSP we did not have items which are in similar nature. They all performed different operations.

It is little difficult to deploy the SSP on servers.

Because there are too many services in one database, so it becomes difficult to scale it. It did not support scaling as we could not add any extra service to the SSP.

Now in SharePoint 2010 SSP has been replaced by Service Applications. These services are not groups under anything, they all run independently. In other sense, service applications provide a-la-carte options to choose from. Per web application, you can configure which service you want to consume.

You can also publish these services outside of the current farm so that these services can also be used elsewhere. You need trust relationship between those farms who wants to consume these services.

You can also write your own services and add that service to this service application.

Here service applications have their own databases unlike shared database in MOSS 2007 SSP.

You can use PowerShell commands to play around with these services.

Get-SPServiceApplication returns all  service applications.

Get-SPServiceApplication-name {servicename} to get the service object.

From there you can get all other properties related to the service.

Bottom line is SSP services are now split into individual services and can be consumed from web applications as and when needed. 

These services are:

Profiles, Audiences -> People Service App

Search -> Search Service App

Performance Point -> Performance Point Service App

Excel -> Excel Service App

Office Web Applications -> Office Web App

Visio Services -> Visio Service App

Word -> Word Service App

PowerPoint -> PowerPoint Service App

Project Server -> Project Server App

Here are some new services that have been introduced in the SharePoint 2010.

Access Services - Allows viewing, editing and accessing Access databases in a browser.

Managed Metadata Service - allows access to managed taxonomy hierarchies, keywords, and social tags for site collections.

Secure Store Service – Provides capability to store data (e.g. credential set) securely and associate it to a specific identity or group of identities.

State Service - Provides temporary storage of user session data for Office SharePoint Server components.

Visio Graphics Service – Helps to view Visio diagrams in a web browser.

Word Conversion Service Application – Allows converting documents into different formats.

I hope this will give you a basic overview about SSP repalcement

How to strengthen Powerusers or Superusers network for SharePoint

Introduction to Superuser network

• A network of expert end users who other employees can go to with questions and problems related to their Intranet system
• But often these programs fade away or never really work due to a few simple reasons
• Either super users retire, quit or simply can’t hold the role anymore, and no one replaces them
• Or the pleas of managers who never really buy in to the system and often tell those employees to get back to their “real jobs,” finally take their toll.

 

SharePoint 15: what to expect in Spring 2013

Speculation is running rampant, and Microsoft has yet to confirm anything, but expect the next version of SharePoint  – dubbed SharePoint 15 – to be released by Spring 2013. We do know for certain, however, that MS has put significant money into redesigning the user interface – the look-and-feel –by a factor of four. Count on a major redesign of the My Sites / My Profiles.

Additionally, we also know that SP15 will have better cloud support and functionality, when compared to the current SharePoint Online offering, better mobile support, and web development support. We can also expect general improvements to all the social tools, such as blogs and wikis (which in previous versions, leave a lot to be desired).

Here’s a summary of the suggested, forthcoming improvements to SharePoint: 

• Improved design interface for social networking
• Better suited for cloud environment
• A new SharePoint app marketplace
• Improved mobile support
• Simplified development and integration
• Overhauled Client Object Model (COM)
• Enable workflow looping in SharePoint Designer, eliminating the need for the Visual Studio
• Cross-platform authentication 

Setup claims authentication through PowerShell

Following script updates the web application to use Claims based authentication:

$webApplicationUrl = "http://intranet"
$webapp= Get-SPWebApplication $webApplicationUrl
$webapp.UseClaimsAuthentication
$webapp.UseClaimsAuthentication=$True
$webapp.Update()
$webapp.ProvisionGl

Workflow Event handlers in SharePoint 2010

In SharePoint 2010, we now have new receivers and one of them is workflow event receivers. We can handle different events with respect to the workflows that triggers in SharePoint.

You can handle multiple events with regards to the workflow. These are:

1)    When workflow is starting
2)    When workflow is started
3)    When workflow is completed
4)    When workflow is postponed

As name suggests you can trap different events of the workflow.

These come handy if you have requirement to start series of workflow based on completion criteria from one workflow to another. If one workflow completes and then you want to start another workflow in chain, that can be done now by trapping these events.

This is only for list or library workflows and not for the site workflow.

Here is an example with code.

///
    /// List Workflow Events

    ///

    public class EventReceiver1 : SPWorkflowEventReceiver
    {
       ///
       /// A workflow is starting.

       ///

       public override void WorkflowStarting(SPWorkflowEventProperties properties)
       {
           SPWeb Web = properties.ActivationProperties.List.ParentWeb;     
           SPList lstTracking = Web.Lists["Track Different Events"];       
           SPListItem item = lstTracking.Items.Add();
           item["Title"] = "Starting the workflow";
           item.Update();      
       }
       ///
       /// A workflow was started.

       ///

       public override void WorkflowStarted(SPWorkflowEventProperties properties)
       {
           using (SPSite site = new SPSite(properties.WebUrl))
           {
               using (SPWeb Web = site.OpenWeb())
               {
                   SPList lstTracking = Web.Lists["Track Different Events"];            
                  SPListItem item = lstTracking.Items.Add();
                   item["Title"] = "workflow has been started";
                   item.Update();
               }
           }
       
       }
       ///
       /// A workflow was completed.

       ///

       public override void WorkflowCompleted(SPWorkflowEventProperties properties)
       {
            using (SPSite site = new SPSite(properties.WebUrl))
            {
                using (SPWeb Web = site.OpenWeb())
                {
                    SPList lstTracking = Web.Lists["Track Different Events"];                

                    SPListItem item = lstTracking.Items.Add();

                    item["Title"] = "workflow has been completed";

                    item.Update();
                 
                }
            }            

       }
    }

You can also handle several completion type enumerations in WorkflowCompleted method that helps you to execute certain code based on the completion criteria.

You can also check completion type enumerations in WorkflowCompleted method.

SharePoint Groups vs. Active Directory Groups

I’ve discussed this topic quite often during the last months. After those discussions I figured out that its more a question when to use what kind of group rather than what kind is better than the other. In this post I just write down some advantages and disadvantages of the group types and let you choose what kind fits better for your needs.

SharePoint Group		Active Directory Group
	Members of this group can be added/removed from within SharePoint. The permission to add or remove users from the group can be delegated to SharePoint users.		Members of this group can be managed within Active Directory. Only Active Directory administrators have the permission to modify group memberships.
	Members of this group can be visible to users.		Members of this group are not visible to users.
	Cannot contain another SharePoint group as member.		Can contain another Active Directory Group.
	Must have a unique name on site collection level. The name is the unique identifier of the group.		Can cause serious problems in lage scale scenarios: A user might only be a member of 1024 Active Directory groups (recoursively). If this number is reached the user is no longer able to log on to Windows. Read the Microsoft documentation for more information.
	Can contain SharePoint users that do not exist in the Active Directory.

Comparison of MS SharePoint Online vs On-Premise

	Hosted or Online	On-Premise
Cost	Zero to minimal upfront cost. Capital is not locked up in software and hardware.	Relatively huge upfront software and hardware expenses plus the cost of personnel to install, configure and maintain.
Management	Our team of SharePoint technology experts constantly builds on experience that they are able to cascade across multiple installs.	Only the largest IT shops don’t suffer when one key employee goes on vacation, takes leave or quits.
Accessibility	Browser based hosted apps are accessible independent of location and time.	Maybe you can access the office from remote, maybe you can’t. Who is going to reboot the server?
Technology	We handle all your Microsoft SharePoint online upgrades. Our support team is prepared and our expertise remains in step.	The cost of the time and effort to remain current can be excessive.
Security	Installing the latest patches and upgrades is naturally one of our core competencies.	Devoting resources to patches are typically given a low priority in many organizations.
Scalability	Scale up—no problem. Or scale down and never have to worry about issuing a pink slip.	More hardware + more software + more staff = more money.
Sustainability	Our staff with SharePoint experience is large enough that there is no noticeable impact from turnover and attrition.	The knowledge and expertise is concentrated in too few hands/heads. You’re one “I quit” away from chaos.
Flexibility	Month to month billing means you can change Microsoft SharePoint Online plans or platforms with ease—no questions asked.	Replacing a software platform can often feel like you’ve paid double.
Payment	Bite-sized month to month fees automatically billed to your credit card.	Large purchases involve meetings, approvals, purchase orders and calls from accounting.
Vendor(s)	A single point of contact. We’re here for you 24/7 via chat, email or phone.	Multiple vendors, multiple support contracts, multiple bills, multiple headaches.
Service	No long term contract dictates our mission…keep you happy	Fickle technology staff can be a constant challenge to manage.
Redundancy	Hardened Class A data center with backup power, full fire protection, etc.	It can be done but it’s going to cost you twice as much.
Customization	Customization options can be limited with hosted apps. But how often do you require customization anyway?	Customizing requires expertise, plus time & money. In most cases it can’t be justified when compared to Microsoft SharePoint Online.

What are the major advantages of using Active Directory group in SharePoint?

• Security behind AD is intense. Microsoft's entire enterprise of applications all utilize AD for security
• Allows for client integration, so opening word doc from a library will keep the file connected to SharePoint. This is a little more complex with FBA
• Assuming your're using AD for internal users, you can centrally manage all your users in one auth store
• You can use AD groups in SharePoint
• Easier management of single sign on and BDC connections (if you're using them)
• Active Directory's real benefit lies in Domain management and Integration with other programs (particularly ones like exchange)

What happens behind the scene when we create new Web Application in SharePoint?

When we create a new Web Application in SharePoint, following are various actions which happens behind the scene:

• Creates a unique entry in SharePoint configuration DB for the Web Application and assign GUID to that entry.
• Create and configures a new site in IIS
• Creates and configures a new IIS application pool.
• Configures authentication protocol and encryption settings.
• Assign a Default alternate access mapping for the Web Application.
• Creates the first content database for the Web Application.
• Associate a search service with the Web application.
• Assign a name to the Web application that appears in the Web application list in SharePoint Central Administration.
• Assign general settings to the Web application, such as maximum file upload size and default time zone.
• Updates the web.config file to make entries for custom HTTP Modules and Handlers for SharePoint.
• Creates virtual directories for SharePoint web services and SharePoint layouts folder.

After creating a Web Application in SharePoint, the web site is actually not created yet. It means if you try to access the Web Application using the web app url, it will show you "Page cannot be displayed" error. Basically at this point of time, a web application has been created and all the mandatory configuration has been done. Now the next step is to create a Site Collection using a particular Site Definition, then only the actual site will be created and you will be able to access the site using the url of Site Collection.

SharePoint 2007 Hotfix – KB936867

We had plan to cleaning up a team site getting rid of things that were no longer needed and lists that were no longer used. Well, it just so happens one user executed a click path that sent about 23 of her 78 sites in to the ether and whenever she or her teammates tried to access the sites they would get a plain SharePoint "500 – Internal Server Error" page.

I checked the logs and couldn't find anything in the server error.

After some looking around in the Webs table of the content database for the SharePoint web application, I noticed that the sites were not deleted or missing. So this pointed out to me either we had an IIS problem or an internal SharePoint problem. Since much of the configuration is stored in the database for SharePoint I was sure that this is a SharePoint problem. Especially since we were getting a plain 500 Error page from SharePoint, not like the one that IIS would put out if it had run in to a problem.

Finally I called to Microsoft Support after working through all of my known debugging techniques. The call started with us redoing much of the checks I had already performed, but then the support tech mentioned there is a forthcoming KB article that would mention a Hotfix for SharePoint that would not be publicly available, but you would have to get in touch with Microsoft Support to get it.

The KB article that he was talking about was KB936867. According to him it is in process of being published. The hotfix is supposed to fix about 11 different issues that Microsoft has been seeing with SharePoint that has been the highest on the radar of issues.

Before we started the Hotfix process, I had mentioned to the support technician that the farm we were working with was a B2 installation that was migrated to B2TR migrated to RTM. This was not a concern to him. Nevertheless we started the Hotfix install. Hotfix installed fine, then we needed to run the Configuration Wizard to apply the hotfix to the SharePoint farm. Half way through the applying of the hotfix, the configuration wizard bombs and tells us that half of the SharePoint Database is now upgraded and the other half is not. The 2 databases the blew up was the Shared Services Provider database and the Configuration database.

So now we have ½ of the farm database upgraded and the other ½ not.

Needless to say after some major panic and working through a few different scenarios, the support tech finally gets a product guy in from the Hotfix or SharePoint team (not sure which), but from what I gained from talking to him is that we shouldn't have used this hotfix if this installation was migrated from the Beta 2 bits. Oooops.

From that moment on, it was pretty apparent that I would be rebuilding my SharePoint farm and reconfiguring my web applications, reconfiguring my Shared Service Provider and going through general hell. And we ended up having to recreate the Configuration database and the Shared Service Database.

Lesson to be learned from this. Never ever only ask once if your farm being migrated from beta bits will be a problem. Make sure that who you are talking to knows for sure that this has been tried and proved.

So, if you run across someone that is mentioning to you that you need to install this hotfix be VERY sure that you are not running the hotfix on a Beta 2 migrated installation. Or be sure that it mentions that this is an approved use of the hotfix. If not, you might be running in to a situation like I did and have to go through hell to get your farm backup and running.

Hopefully this will keep you from running in to the same situation.

SharePoint 2010 vs. MOSS 2007

SP 2010	MOSS 2007
Look and feel
In SP 2010 look and feel perspective there will be a ribbon where we can have a look and feel like Office 2010	In MOSS 2007 there is no ribbon
Deployment of Web parts
In SharePoint 2010 deploying custom web part is pretty simple i.e. just right click on the solution and click Deploy	In MOSS 2007 you need to drag the dll either to bin or GAC
Silverlight Application
In SP 2010 we can create a Silverlight application directly from Visual Studio 2010	In MOSS 2007 we have to create a web part to host Silverlight application
Shared Database & Service Application
In SP 2010 there is no SSP but there is a concept of Service Application like BCS as one service application, Excel Services as another service application, User Profile as separate service application General idea is that you have an application for each service, rather than one application with lots of service crammed into it Own database rather than shared database in SP 2010	In MOSS 2007 we have SSP where we can work around with BI,Search Settings, User Profile Import, Excel Services, Info path In Database also we use to have separate area for SSP stuff
Easy exports/imports between the forms
In SP 2010 we can update existing information	In MOSS 2007 through we can just read the information and we can't update the existing services
Improvement in Deployment
In SP 2010 we can Deploy through Farm based and solution based solution in SP 2010	In MOSS 2007 there is no such option
Alerts
In SP 2010 it has been improved in validation and unique values. While creating column itself we have an option "Allow Duplicate values" to Yes or No	In MOSS 2007 alerts were sent only through emails but in SP 2010 users can also send alerts to mobile device as SMS message. A New property delivery channel is introduced to indicate, whether the alerts is delivered as Email or an SMS message
Improvements of events
New events for list creation and web creation	No List and web events in MOSS 2007
Getting Items from the list
In SP 2010 through object model we can fetch multiple list data by LINQ query and object model	In MOSS 2007 we can fetch only through object model
Rating
In SP 2010 we can have rating column by default	In MOSS 2007 we should install the feature that is available in codeplex to have rating
Key Word Suggestions
In SP 2010 we can have keyword suggestions	In MOSS 2007 we don’t have any keyword suggestions
Taxonomy
In SP 2010 we can create Taxonomy by using Managed Metadata service	In MOSS 2007 we don’t have taxonomy
Other Features
In SP 2010 we have Power Shell Scripting, JavaScript object model, Chart Web Parts	In MOSS 2007 we don’t have Power Shell Scripting, JavaScript object model, Chart Web Parts
Running stsadm command
In SP 2010 we have to go 14 hive path to run stsadm command	In MOSS 2007 we have to go 12 hive path to run stsadm command

Classic Authentication and Claim based authentication

What are the differences between Classic Mode Authentication and Claims based Authentication?

Classic Mode Authentication: This refers to the integrated windows authentication. We cannot configure the forms based authentication if your web application is using Classic Mode Authentication. We can convert a web application from Classic Mode Authentication to Claims Based Authentication.

However, this can only be done using PowerShell commands and it is an irreversible process.

Claims Based Authentication: SharePoint 2010 is built on Windows Identity Foundation. It enables authentication from windows as well as non-windows based systems. This also provides the capability to have multiple authentication in a single URL.

SharePoint 2007 - Master Merge

The concept of  Master merge and how indexing happens behind the scene in SharePoint

• Index server when crawls the content it creates a shallow index, shallow index is a smaller part of our index file. The reason for creating Shallow index, we can write efficiently on a smaller file rather on a bigger file and also we can propagate the index file faster on the network to all query servers.
• Reading all smaller index file will be time consuming, example when we server search query : "SharePoint" it would be a larger overhead to open all these smaller index file and perform search we need more file I/O. To prevent this overhead we perform merging of all these smaller files to a one single file called "Master Index" so that we can open a single file and perform search on the single index file. This process is called Master merge, merging all your shallow index to master index.
• By default we don't have any time period to schedule Master merge, it happens when we have shallow index more than 10% size of master index
• Master merge happens at query servers / index servers
• We can force master merge or change the size limit below is the link written by Bill Baer (very interesting) http://blogs.technet.com/wbaer/archive/2007/12/03/managing-master-merge-in-microsoft-office-sharepoint-server-2007.aspx

PowerShell

If you would like a complete listing of the cmdlets:

Get-Command -module Microsoft.SharePoint.PowerShell | format-table name > textfilename.txt

SharePoint security vulnerability and patch

On Microsoft's security bulletin, a SharePoint security vulnerability and patch was announced: http://technet.microsoft.com/en-us/security/bulletin/ms11-074.  The specific issue at hand is XSS related, allowing for malicious URL's to execute SharePoint commands unintentionally, and overall is bad news.  Of course, most modern browsers have XSS protection built in, but most also disable it when in " Intranet" mode (which is how most SharePoint deployments tend to be deployed)

This security issue affects Office 2007 and 2010 clients, as well as SharePoint 2007 and 2010 servers.  Installing this update for Office clients is pretty straight forward, but like all SharePoint related updates, this one has some issues.

if you are running SharePoint 2010 or 2007, you install these updates as soon as you can.  If you don't install them, WSUS may install them for you and wind up breaking your farm.  BUT, make sure you thoroughly test them before you do anything on production.

If you're running SharePoint 2010 I recommend doing the following:

If you aren't already on SharePoint 2010 Service Pack 1 + the June Cumulative Updates, go ahead and do so now.  Many of the SP1 issues that were reported regarding Claims authentication are resolved in the June 2011 update.
Ensure your environment is still functioning post-Service Pack and CU.  Resolve any issues that may arise (such as restarting the User Profile Service, and ignoring the false farm health check error Microsoft discovered)
Install all of the SharePoint 2010 security updates:
SharePoint Foundation 2010 "base" update –
KB 2494001
KB 2494022
KB 2560885
KB 2560890
KB 2566456
KB 2566954
KB 2566958
KB 2566960
If you're running Office Web Apps – KB 2566449
Make sure that you've run the configuration wizard on all servers in your farm.
Like all SharePoint updates, TEST IT BEFORE YOU INSTALL IT IN PRODUCTION.  If you only deploy partial versions of this update, there are chances your publishing pages or User Profile service may throw errors until you get everything back in sync.

Office Communicator integration in a custom webpart for SharePoint 2010

With SharePoint 2010's new social capabilities, I see this integration as being a must-have for any webpart or other custom interface that refers to people.

How does SharePoint do it?

Whenever a person's name is mentioned in the standard SharePoint UI, if the user has Office Communicator installed, the name will be shown with the Office Communicator 'pawn'. The pawn shows the user's status and gives them access to the pop-up menu to see more details.

Depending on which version of Office the user has installed, the experience will vary, however it will be the same experience as seen in Outlook. In Office 2010 it is a square icon with a richer drop-menu, in 2007 it is round.

In addition to the pawn, the user's name will be a hyperlink to their profile page. This will vary depending on whether the user is in the main 'User Profile Service Application' (or 'profile database' for old-school terminology) or not. If the user does have a profile then the hyperlink will redirect to the user's main profile page under My Site. If the user does not have a profile, the hyperlink will redirect to the SharePoint Foundation 'User Information' page. All users in a site have a 'User Information' page, if they also have a User Profile the settings from the profile are synchronised down to the User Information page on a scheduled basis (by a timer job).

The key piece of information to make the presence work is the user's SIP Address which is basically their IM address (This is not always the same as email address). When a users is either added to a site in SharePoint or has their profile imported, the SIP Address will be drawn from Active Directory which is where OCS stores it and placed into the SIP address field in either the user's profile which will in turn synchronise down to the site's local 'User Information' page.

How does the pawn work in terms of HTML?

The pawn is basically an IMG element which has "IMNRC('[user's SIP Address]')" for the onload function. This will user client-side script that is part of Office to load the presence pawn on the page.
The hyperlink on the user's name is just a simple A element which redirects to "/_layouts/userdisp.aspx?ID=[User's local user information list item ID]". This userdisp.aspx page will then redirect to the user's main profile page if they have a profile, otherwise it will display the basic information that the user information list item stores.

How to get the SIP Address?

Hopefully you'll have spotted by now that the presence pawn relies on the user's SIP Address. To get the SIP address you need to get the user's User Information list item from the local site's (SPWeb to be precise) SiteUserInfoList which is basically a hiddenSPList that is in every web.
There is a handy property to get you to this list called 'SPWeb.SiteUserInfoList'. This will give you an SPList object which represents the User Information list. From here you just need to find the item that represents the user you wish to display. The best way to do this is via their ID (the ID of the list item) by calling SPWeb.SiteUserInfoList.GetItemById([int User's ID]), however you can also use a variety of other methods which use SPQuery or match a specific field to a value.
In most scenarios, you may be getting the user information from a SPFieldUserValueCollection which is basically the field type that is used for 'Person' fields. If this is the case you can use SPFieldUserValue.LookupId to get the ID of the user's User Information list item.

Putting it all together

This code sample is a method that accepts an SPFieldUserValueCollection and SPSite as inputs and returns back the HTML for displaying each entry in the SPFieldUserValueCollection with the presence pawn and link to their profile. This will be presented exactly as SharePoint presents it by default. This could be extended to use ALT tags in the IMG and A elements.
I then simply add the HTML to an HtmlWriter or in my case a TableCell.Text property to display it on screen. I've take a few extra steps here by adding ID and alt tags and trying to make the code readable, but I'm sure you get the idea:

What really happens in the background when a user performs a SharePoint search

Below is the diagram that gives you a pictorial vision of the high level communication that happens between IIS, MSSearch.exe, the Search DB and the Full-Text Index.

Below is a breakdown of the 6 different stages.

Stage 1:

First thing that you need is the search query. The WebPart or the Web Service helps build up a search query. This search query is then passed to the Query Object Model (OM) via the WebPart or the Web Service.

This step happens between the client machine that submits the query and the web front end. Network traffic would show the following GET and the response:

Request:

GET /searchcenter/Pages/Results.aspx?k=sharepoint&s=All%20Sites HTTP/1.1Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*Accept-Language: en-usUA-CPU: x86Accept-Encoding: gzip, deflateUser-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)Host: wfeConnection: Keep-AliveAuthorization: NTLM TlRMTVNTUAADAAAAGAAYAHQAAAAYABgAjAAAAA4ADgBIAAAAGgAaAFYAAAAEAAQAcAAA
AAAAAACkAAAABYKIogUCzg4AAAAPQwBPAE4AVABPAFMATwBBAGQAbQBpAG4AaQBzAHQAc
gBhAHQAbwByAEQAQwApEBdASjCmUQAAAAAAAAAAAAAAAAAAAABd0mXyd+vSlujhEVc5rpV dPtQYe1WTXa0=Cookie: WSS_KeepSessionAuthenticated=80; MSOWebPartPage_AnonymousAccessCookie=80

Note (Highlight): In the above GET request ‘k’=keyword used and‘s’=scope

Response:

HTTP/1.1 200 OKCache-Control: private, max-age=0Content-Length: 76654Content-Type: text/html; charset=utf-8Expires: Tue, 08 Dec 2009 22:33:34 GMTLast-Modified: Wed, 23 Dec 2009 22:33:34 GMTServer: Microsoft-IIS/6.0X-Powered-By: ASP.NETMicrosoftSharePointTeamServices: 12.0.0.6318X-AspNet-Version: 2.0.50727Set-Cookie: WSS_KeepSessionAuthenticated=80; path=/Set-Cookie: MSOWebPartPage_AnonymousAccessCookie=80; expires=Wed, 23-Dec-2009 23:03:34 GMT; path=/Set-Cookie: http%3A%2F%2Fwfe%2FSearchCenter%2FDiscovery=WorkspaceSiteName=U2VhcmNo&WorkspaceSiteUrl=aHR0cDovL3dmZS9TZ
WFyY2hDZW50ZXI=&WorkspaceSiteTime=MjAwOS0xMi0yM1QyMjozMzozNQ==; expires=Fri, 22-Jan-2010 22:33:35 GMT; path=/_vti_bin/Discovery.asmxDate: Wed, 23 Dec 2009 22:33:34 GMT

Stage 2:

The query Object Model then in turn calls the Query Processor. The role of the Query Processor is to join results from the Full-Text Index with the SearchDB.

On SQL the two databases that are queried are:

• SSP database.
• SSP Search database.

On the SSP database proc_MSS_GetKeywordInformation is run to get the keywords for displaying on the site if Office SharePoint Search is being used.
On the search database the proc_MSSGetMultipleResults is run to get the properties from MSSDocProps table. Both these stored procedures and more details about them can be viewed via a SQL trace.

Stage 3:

The Query processor then opens a Query Pipe to the query machine. The link between the Web Front End and Query server is a query pipe only provided by MSSearch.exe. When the Query pipe network traffic is examined, it is revealed to be SMB traffic. Network Monitor 3 will identify this traffic as being the CIS protocol

A high level overview of what we can see via a Network Monitor capture is below.

The initial communication from the WFE to the Query Server occurs via one of the random (1024 to ~65000) WFE ports. This is done to Query ports 139 and 445 and finally establishing the connection through SMB.

In the network capture you will also notice a 'Tree Connect AndX Request' for path \\QueryServer\IPC$ from WFE's random port to the Query machine's port 445. Port 139 is used as the end point mapper.

Once the Tree is created, the path is changed to \OSearch in order to be able to query the flat files inside the Query server's file system.
Note however that at this point the default 'searchindexpropagation' share which is used while propagating index from index server to the query server is not used to retrieve the query responses.

Stage 4:

The indexer plug-in (refer diagram) on the query machine retrieves the results from the index. The Indexer Plug-in is the only part of Query that will access the Full-Text Index. We will get into more details about this in the following blogs of the same series. .

Stage 5:

Now let’s get into the part where the results are returned. The results come back from the Full-Text Index as DocIds. This task occurs based on the data that the web front end receives back from the Query and the SQL servers.

Stage 6:

The end result is then the function of the query processor (refer diagram above). The Query processor joins results from the Search DB and full text index. It takes the DocIds and does the join to the SearchDB to access the document properties (Title, Display URL, doc format, size, etc). This task occurs based on the data that the web front end receives back from the Query and the SQL servers.