Search, Upgrades, and DB Maintenance

Recently I have run into some issues with Search. Indexing Errors, Propagation Errors, Account Errors, etc. Accounts that have been imported for months couldn't search items that were newly created and newly imported accounts could search, some search items were showing up immediately after the incremental crawl, while others that had been uploaded and checked-in for quite some time and still failed to show up, etc., etc... After about a week of research and some mild head bashing I have come up with some solutions as well as some proactive maintenance and scheduling plans that should help you out.

Here's what I did:

First off, install the latest infrastructures upgrades. There are two; one for WSS and one for MOSS. Make sure you implement the WSS one first and the MOSS one Second. However, make sure you implement them both! The WSS upgrade will fix some things and make a few improvements but the MOSS Infrastructure upgrade will give you a whole new feature set and it's own administrative console. For all of the updates see Microsoft's article http://blogs.msdn.com/sharepoint/archive/2008/07/15/announcing-availability-of-infrastructure-updates.aspx

Once the upgrades are in place its time to review the second part; the Search schedule. It was brought to my attention (with a call to Microsoft regarding another matter) that a full crawl everyday for a large SharePoint implemtation is not necessary and more than likely counter productive. Perform a Full Crawl once a week on a Friday or Saturday morning. Perform incremental crawls 2 times a day (at most) in the morning and the evening.

Next we move on to the SQL/Database side of things where there are also a few recommendations. The first has to do with the Transaction Logs. The TL's are used to record any and all modifications made to databases, including the search/index database. Make sure that your TL's have enough space and are growing accordingly with the rest of your farm.
After all of this, you should continue to perform routine database maintenace as well as a few other things. Once a month you or your DBA's should defrag your SharePoint Databases just to keep them organized and dust free. One other thing to consider doing monthly is Rebuilding and Reindexing your SharePoint Search. This is again to de-clutter, clean up, and start fresh.

To Summarize:

• Install WSS 3.0 Infrastructure Upgrade
• Install MOSS 2007 Infrastructure Upgrade
• Run a Full Crawl Once a Week
• Run 2 Incremental Crawls daily
• Ensure your Transaction Logs have space to grow and grow accordingly with your farm
• Perform database maintenance regularly
• Once per month Perform SharePoint DB Defrag
• Once per month (or two months) Perform a complete Rebuild/Reindex of SharePoint Search

-Brian Grabowski

Migrations, Site Definitions, and MergeContentDBs

When migrating from SPS 2003 to MOSS 2007 we all know there are 3 options to pick from;

In-Place (Quickest, most room for error)

Gradual and (Ease of use and control)

Database (More manual, ease of use, leaves old farm completely in tact)

For all intents and purposes this post will summarize a Database Migration and Resetting Site Definitions, but the focus is primarily on the stsadm command mergecontentdbs and how I used it to restore multiple site collections at once rather than using backup and restore.

Here's what I did:

Backup and Restore (from SPS 2003 to Development MOSS 2007)

• Put Content DB's in Read Only Mode
• Run Pre-Upgrade Scan
• Reset to original Site Definition
• Backup Content DB's
• Transfer .bak files to Development
• Restore DB's in Development SQL Environment

Upgrade and UAT (In Development MOSS 2007)

• Attach newly upgraded datbase(s) as new content database(s)
• Upgrade Process (Automatic)
• User Acceptance Testing

Restore (to Production MOSS 2007)

Once sites have passed UAT and all functionality can be accounted for, sites will be “merged,” or moved to a new content database that is ready to be attached in the production environment. The new database will be backed up and then restored as an entire collection. With this method we can move multiple sites at once rather than manually moving each site collection one by one. The following command line prompt is an example of merging content databases;

First, the option that I will be showing here (#3) requires a file to read from. That file is a list of the sites in the database that you want to move to another one. This list can be edited to exclude certain sites. In our case excluding sites that haven't passed UAT.

Example;

stsadm -o enumsites -url http://sharepoint/ -databasename Content_DB1 > c:\mergesites.txt

Second, perform the mergecontentdbs command.

stsadm -o mergecontentdbs
-url
-sourcedatabasename
-destinationdatabasename
[-operation] {1-3}
1 - Analyze (default)
2 - Full database merge
3 - Read from file
[-filename]

Example;

stsadm -o mergecontentdbs -url http://sharepoint/ -sourcedatabasename Old_DB1 -destinationdatabasename New_DB1 -operation 3 -filename c:\mergesites.txt

The new content database will be restored to the Production SQL Environment and will then be attached to the SharePoint Farm.

• Restore new merged and backed up database to Production SQL Environment
• Add New Content Database in Production SharePoint Farm

-Brian Grabowski

Searching for Numbers in SharePoint

If you have been trying to search for a number or a combination of numbers in SharePoint such as an IP address or strictly version numbers, you have probably been coming up short. This is due to "Noise Filtering." There is a text file that filters noise words such as "a," "the," "and," numbers, etc.

Here's what I did:

Edit the noiseenu.txt file
Go to "Sharepoint Portal Server Install directory"/Applications/Guid/Config/noiseenu.txtRemove the "numbers" from the file
Restart the SharePoint Search Service
Reset the search indexes and perform a full crawl

Once the crawl is finished you should now be able to search for numbers.

-Brian Grabowski

Adding AD Groups to SharePoint

When adding AD Groups to SharePoint some groups show up while others do not. The case of the matter here depends upon what type of group you are trying to add. Distribution Lists cannot be added only Security Groups can be added.

• Ensure you are using Security Groups and not Distribution Groups. A Distribution group can not be listed in the DACL and can not be used to assign permissions againstDistribution


• Lists do not contain the user object, only an "alias"


• Ensure Security Groups are populated with at least one account


• Create your Active Directory Groups


• Add at least one user to each group (and make sure that one user will have their user profile imported) Run a full user profile import in SharePoint Create your audiences (you should now see all the groups that have members with profiles)

-Brian Grabowski

The search request was unable to connect to the Search Service

If Search was functioning fine yesterday, but seems to have taken a crap on you today you might want to try these quick and easy fixes followed by some maintenance on Service Accounts and their Logins.

Here's what I did:

Restart these services:

Indexing Service
Office SharePoint Server Search
Windows SharePoint Services Search

After this do a quick IISReset and test the search functionality.

Or heres a quick script to restart all of the services and perform an IISReset for you.

net stop "Indexing Service" && net start "Indexing Service"
net stop "Office SharePoint Server Search" && net start "Office SharePoint Server Search"
net stop "Windows SharePoint Services Search" && net start "Windows SharePoint Services Search"

iisreset \\servername /noforce
iisreset \\servername /status
pause


If this worked for you then you might want to check the Service Accounts that you have tied to these services. They stopped or quit working for a reason. Investigate your Search and Query Server's Event Viewer and see what happened.

-Brian Grabowski

SharePoint Latency and Optimiaztion over WAN

Colonel Sanders: Prepare ship for lightspeed!
Dark Helmet: No, no, no! Lightspeed's too slow!
Colonel Sanders: Lightspeed too slow?
Dark Helmet: Yes. We're going to have to go right to.......Ludicrous speed!

One key to an efficient workforce is providing pertinent information in a reliable and quick manner. SharePoint is an excellent foundation for this theory. However, if you have ever wanted to boost speed and increase performace, here is one option that I have used that may give you that extra umph! (THE FOLLOWING REQUIRES REGISTRY CHANGES TO SERVERS...TEST FIRST!!!)

RFC 1323

RFC 1323 is a high-speed transmission path that defines window scaling that allows a receiver to advertise a window size larger than 65,535 bytes.

"A TCP Window Scale option includes a window scaling factor that, when combined with the 16-bit Window field in the TCP header, can increase the receive window size to a maximum of approximately 1GB. The Window Scale option is sent only in synchronize (SYN) segments during the connection establishment process. Both TCP peers can indicate different window scaling factors to use for their receive window sizes. By allowing a sender to send more data on a connection, TCP window scaling allows TCP nodes to better utilize some types of transmission paths with high BDPs."

So in short it allows more information to be sent and recieved, ultimately speeding up performance. In our lab we were able to take an 8MB file that was previously downloading at around 10 seconds to just over 2 seconds.

Pre RFC 1323 tuning= 8MB/10 sec.
Post RFC 1323 tuning= 8MB/2 sec.

Here's what I did:

(The following registry keys may or may not exist...create or edit them...also keep in mind that you can play with the values, however don't increase the sizes so much that they overload the Processor with heavy I/O loads.)

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\AFD\Parameters]

"DefaultSendWindow"=dword:00100a40

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters]

"TcpWindowSize"=dword:00100a40

"GlobalMaxTcpWindowSize"=dword:00100a40

"Tcp1323Opts"=dword:00000003

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HTTP\Parameters]

"MaxBytesPerSend"=dword:000fffff

-Brian Grabowski

STSADM, Site Template Codes, and Scripted Site Creation

If ever you've wanted to create Sites from the command line and quit halfway through because you had no clue what the Templates were called or named, you can finally press enter and watch impatiently for the cursor to blink down and say success. Keep in mind these are only the inherent MOSS 2007 Templates and not anything custom that you've thrown in there.

Here's the list:

WSS Templates

Team Site: STS#0
Blank Site: STS#1
Document Workspace: STS#2
Wiki Site: WIKI#0
Blog Site: BLOG#0
Basic Meeting Workspace: MPS#0
Blank Meeting Workspace: MPS#1
Decision Meeting Workspace: MPS#2
Social Meeting Workspace: MPS#3
Multiple Meeting Workspace: MPS#4

MOSS Templates

Document Center: BDR#0
Site Directory: SPSSITE#0
Report Center: SPSREPORTCENTER#0
Search Center with Tabs: SRCHCEN#0
My Site Host: SPSMSITEHOST#0
Search Center: SRCHCENTERLITE#0
Personalisation Site: SPSMSITE#0
Collaboration Portal: SPSPORTAL#0
Publishing Portal: BLANKINTERNETCONTAINER#0
Publishing Site: CMSPUBLISHING#0
Publishing Site with Workflow: BLANKINTERNET#2
News Site: SPSNHOME#0

All you need is the title and the code.

Example;

-sitetemplate STS#0

Also, here is a little code snippet in a batch file that will ask the correct questions, input the data, and create a Site Collection for you without having to go through CA. (Just edit the code to match your configuration.)

@ECHO off
cls
:start
ECHO.
ECHO Please Enter the following information:
set/p url=Complete the Site Url you wish to create. http://mysharepointsite/sites/
set/p owneremail=Enter the Owner's Email address.
set/p ownerlogin=Enter the Owner's Account Name.
set/p sitetemplate=Enter the SiteTemplate Code you wish to use.
set/p title=Enter the Title of the Site.
ECHO 1. Would you like to create the Site Collection?
set choice=
set /p choice=[Y/N] if '%choice%'=='Y' goto CreateSite
if '%choice%'=='N' goto start
ECHO "%choice%" is not valid please try again
ECHO.
goto start
:CreateSite
stsadm.exe -o createsite -url http://mysharepointsite/sites/%url% -owneremail %owneremail% -ownerlogin %ownerlogin% -sitetemplate %sitetemplate% -title %title%
goto end
:end
pause


-Brian Grabowski