Monday, November 24, 2008

Migrating SharePoint 2003 PWA Sites to MOSS 2007

Migrating SPS 2003 to MOSS 2007 can be tricky to begin with. Now throw in Project Server 2003, PWA Sites, and templates that don't exist in your MOSS 2007 environment. All is not lost and it's actually easier than you might think. In this scenario we again will be using the database upgrade method. (This entire scenario is based upon the assumption that you do not and probably will not have Project Server 2007 installed with your MOSS 2007 deployment.)

Here's what I did:

So let me set up the scenario. You have;
SPS2003 with Project Server 2003 and a mix of default templates and PWA templates
MOSS 2007 with NO Project Server and the same mix of default templates and PWA templates that need to be migrated into MOSS 2007

What you will need; (for a database migration)
SPS 2003
SQL 2000/2005
Winrar, Winzip, etc...(compressing backed up databases before you move them decreases the risk of corrupting your databases)
Project Server 2007 (Trial or Licensed...A trial version will work on a server with other trial licenses. You can not mix and match trial versions with licensed versions.)
MOSS 2007

On to the fix;

BEFORE YOU PERFORM ANY OF THESE STEPS MAKE SURE YOU HAVE CLEAN BACKUPS OF EVERYTHING SHAREPOINT!!!

Install Project Server on a Development Server or VM with MOSS 2007 installed
Run PSConfig
This Reconfigures the Farm to work with Project Server 2007
Installs Templates, Files, Data
It will install key files to;/config/upgrade/pwsupgrade.xml
/template/1033/xml/webtemppwa.xml
/template/site templates/PWA and PWS
/template/features/PWA, PWAProposals, PWS, PWSCommitments, PWSCTypes, PWSocLibs, PWSFields, PWSIssues, PWSRisks
Copy Installed Files
Uninstall Project Server 2007
(You can leave it installed if you wish, but if you want to uninstall it continue to the next step)
(Uninstalling Project Server 2007 kills the SharePoint Farm…nothing in IIS, no CA, nothing…all you have left is DB’s in SQL)
***IMPORTANT***
At this point SharePoint is no more, you won't be able to access sites or CA
***DON'T WORRY***
IISReset
Move copied files to correct locations on your Production Server…/upgrade, /xml, /site templates
Re-run PSConfig (THIS WILL BRING YOUR SHAREPOINT ENVIRONMENT BACK)
Connect to existing Farm
Reattach SQL Server
Reattach Config_DB
Run through PSConfig Steps 1-9
IISReset
ReDeploy Solutions (the Uninstall will retract all deployments)
Restore Migrated DB to Development
Attach Migraged DB to Development
At this point in time you should now be able to browse to your SPS 2003 PWA Sites in your Development MOSS 2007 environment
If everything seems fine in Development restore and attach to Production

-Brian Grabowski

Monday, November 10, 2008

SharePoint Backup Models

Types and Descriptions of Backup Models as suggested by Microsoft.

"Full Backup Model w/ Truncation
The full recovery model uses log backups to prevent data loss in the broadest range of failure scenarios, and backing and restoring the transaction log (log backups) is required. The advantage of using log backups is that they let you to restore a database to any point of time that is contained within a log backup (point-in-time recovery). Assuming you can back up the active log after a disaster occurs; you can restore the database up to the point of failure without data loss. The disadvantages of using log backups are that they require storage space and increase restore time and complexity.

Truncation
Under the full recovery model or bulk-logged recovery model, the inactive part of the log cannot be truncated until all its log records have been captured in a log backup. This is needed to maintain the log chain—a series of log records having an unbroken sequence of log sequence numbers (LSNs). The log is truncated when you back up the transaction log.

Optional Backup Models

Simple Backup Model
The simple recovery model provides the simplest form of backup and restore. Backup is easy to manage because the transaction log is never backed up. However, if there are no log backups, a database can be restored only to the end of the most recent backup of the data. If a failure were to occur, updates that are made after the most recent backup of the data are lost.

Bulk Logged Backup Model
This topic is relevant for optimizing bulk operations on SQL Server databases that typically use the full recovery model.

The bulk-logged recovery model is a special-purpose recovery model that should be used only intermittently to improve the performance of certain large-scale bulk operations, such as bulk imports of large amounts of data. Much of the description of backup under the full recovery model also applies to the bulk-logged recovery model. This topic looks only at considerations that are unique to the bulk-logged recovery model." -Microsoft

-Brian Grabowski

Growth Model for Site Collections and Content Databases

When to Create a New Site Collection

A site collection is a group of sites built on Microsoft Windows SharePoint Services that all exist under a top-level site. To make managing the sites and their content more convenient, you can assign users to be site collection administrators or site collection owners. These are permission levels to give to users who you want to have full administrative rights to all sites and content within a site collection.

Site Collections should be created for;
New Top Level Sites
New Groups
New Divisions
Any new or non existing major heading or internal site

Examples
IT
Finance
HR
Sales
Marketing

Site Collections should NOT be created for;
Personal Sites
Project Sites
Any minor heading or internal site that should fall under one of the larger more generic, “encompassing” Site Collections

To create Site Collections, Administrators can use;
Central Administration GUI
Application Management
Create Site Collection
STSADM
stsadm.exe -o createsite -url http://myserver/sites/site1 -ownerlogin -owneremail someone@example.com -ownername

When to create a New DB

By default Site Collection are created in the same or last created Content Database unless specified other wise. Content Databases are the warehouses for a Site Collection’s Files, Documents, List Items, Subsites, and overall general data contained within the Site Collection.

New Content Databases should be created for/when;
A Site Collection contains/will contain highly sensitive information that if lost must be backed up as quickly as possible
A current Site Collection is approaching the 100 GB limit (or the pre-determined limit)
A Database is approaching or has hit its maximum number of sites allowed

Limit content database size to enhance manageability
Plan for database sizing that will allow for manageability and performance of your environment.

In most circumstances, to enhance the performance of SharePoint, the use of content databases larger than 100 GB is not suggested. If your design requires a database larger than 100 GB, follow the guidance below. (100 GB is a suggestion only from a SQL Manageability and Usability standpoint. SQL 2005 is “capable” of handling Terabytes of data, however managing a database that large is not feasible. To determine the actual limit or cut off points for your SQL environment speak with the DBA’s and see what they are comfortable with managing and how the current backup model is setup.)

Use a single site collection for the data.
Use a differential backup solution, such as SQL Server or Microsoft System Center Data Protection Manager, rather than the built-in backup and recovery tools.
Test the server running SQL Server and the I/O subsystem before moving to a solution that depends on a 100 GB content database.
Split content from a site collection that is approaching 100 GB into a new site collection in a separate content database to avoid performance or manageability issues.
Limit content databases that contain multiple site collections to approximately 100 GB.

STSADM Command Quick Tip:
To create a new Site Collection in a new Content Database use the following command with the appropriate parameters;
stsadm -o createsiteinnewdb

-Brian Grabowski

Tuesday, November 4, 2008

Capacity and Scaling

Scalability is the ability of an application to efficiently use more resources in order to do more useful work. For example, an application that can service four users on a single-processor system may be able to service 15 users on a four-processor system. In this case, the application is scalable. If adding more processors doesn't increase the number of users serviced (if the application is single threaded, for example), the application isn't scalable.
In regards to SharePoint to accommodate greater user load, add Web Front End Servers. To accommodate greater data load, add capacity to the database server role by increasing the capacity of a single (clustered or mirrored) server, by upgrading to a 64-bit server, increasing Processing Power, increasing RAM, or by adding clustered or mirrored servers. Maintain a ratio of no greater than eight Web server computers to 1 (clustered or mirrored) database server computer.

Scale Up
Scaleup means scaling to a bigger, more powerful server—going from a four-processor server to a 16-processor or 32-processor server, for example. This is the most common way for databases to scale. When your database runs out of resources on your current hardware, you go out and buy a bigger box with more processors and more memory. Scaleup has the advantage of not requiring significant changes to the database. In general, you just install your database on a bigger box and keep running the way you always have, with more database power to handle a heavier load.

Scaleout
Scaleout means expanding to multiple servers rather than a single, bigger server. Scaleout usually has some initial hardware cost advantages—eight four-processor servers generally cost less than one 32-processor server—but this advantage is often cancelled out when licensing and maintenance costs are included. In some cases, the redundancy offered by a scaleout solution is also useful from an availability perspective.

-Brian Grabowski

Friday, October 17, 2008

SharePoint 403 Forbidden Error

During recent migrations we used the database migration method to move SPS 2003 Sites into a MOSS 2007 Farm. The basic process went like this;

  • (SPS 2003 Farm)
  • Prescan
  • Set DB's to read-only
  • Backup DB's
  • Move DB's to MOSS 2007
  • (MOSS 2007 Farm)
  • Restore DB's
  • Attach DB's

Now view your Sites and all works fine right? Not so much, how about a "403 Forbidden" error. Thats always nice! On the SPS 2003 side the sites had been "locked." This can be done through SPSSiteManager.exe or the Central Administration. Either way they were locked.

Once on the MOSS 2007 side all we had to do was "unlock" the sites. The command has changed a little bit and there were a ton of sites, so we opted to script the command that looked something like this;

Here's what I did:

The actual command is;
stsadm.exe -o setsitelock -url -lock

@echoCd\Cd “C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN”stsadm.exe -o setsitelock -url http://sharepoint/sites/sitename -lock none
Repeat for Site #2
Repeat for Site #3
etc... etc...
@echo off
pause
exit

-Brian Grabowski

Wednesday, October 15, 2008

Wiki Pages not in Search Results

With the addition of a rather large Wiki Site in our SharePoint Farm, users began adding new "posts" like wildfire. However, when they went to search for them, all they got was the link to the main Wiki Page or a document or link that contained the word "wiki." After adding content sources, creating rules, and creating scopes I found the quick and simple fix. Click the box that enables search to crawl .aspx pages!

Here's what I did:

  • Go to Site
  • Site Actions
  • Site Settings
  • Search Visibility
  • Check "Yes" for "Allow this web to appear in search results?"
  • Check "Always index all ASPX pages on this site" for "This site does not contain fine-grained permissions. Specify the site's ASPX page indexing behavior:"



  • Click OK
  • Wait for the next or Start an incremental crawl
  • Search for .aspx Wiki Pages

-Brian Grabowski

Search, Upgrades, and DB Maintenance

Recently I have run into some issues with Search. Indexing Errors, Propagation Errors, Account Errors, etc. Accounts that have been imported for months couldn't search items that were newly created and newly imported accounts could search, some search items were showing up immediately after the incremental crawl, while others that had been uploaded and checked-in for quite some time and still failed to show up, etc., etc... After about a week of research and some mild head bashing I have come up with some solutions as well as some proactive maintenance and scheduling plans that should help you out.

Here's what I did:

First off, install the latest infrastructures upgrades. There are two; one for WSS and one for MOSS. Make sure you implement the WSS one first and the MOSS one Second. However, make sure you implement them both! The WSS upgrade will fix some things and make a few improvements but the MOSS Infrastructure upgrade will give you a whole new feature set and it's own administrative console. For all of the updates see Microsoft's article http://blogs.msdn.com/sharepoint/archive/2008/07/15/announcing-availability-of-infrastructure-updates.aspx

Once the upgrades are in place its time to review the second part; the Search schedule. It was brought to my attention (with a call to Microsoft regarding another matter) that a full crawl everyday for a large SharePoint implemtation is not necessary and more than likely counter productive. Perform a Full Crawl once a week on a Friday or Saturday morning. Perform incremental crawls 2 times a day (at most) in the morning and the evening.

Next we move on to the SQL/Database side of things where there are also a few recommendations. The first has to do with the Transaction Logs. The TL's are used to record any and all modifications made to databases, including the search/index database. Make sure that your TL's have enough space and are growing accordingly with the rest of your farm.
After all of this, you should continue to perform routine database maintenace as well as a few other things. Once a month you or your DBA's should defrag your SharePoint Databases just to keep them organized and dust free. One other thing to consider doing monthly is Rebuilding and Reindexing your SharePoint Search. This is again to de-clutter, clean up, and start fresh.

To Summarize:
  • Install WSS 3.0 Infrastructure Upgrade
  • Install MOSS 2007 Infrastructure Upgrade
  • Run a Full Crawl Once a Week
  • Run 2 Incremental Crawls daily
  • Ensure your Transaction Logs have space to grow and grow accordingly with your farm
  • Perform database maintenance regularly
  • Once per month Perform SharePoint DB Defrag
  • Once per month (or two months) Perform a complete Rebuild/Reindex of SharePoint Search
-Brian Grabowski

Thursday, October 9, 2008

Migrations, Site Definitions, and MergeContentDBs

When migrating from SPS 2003 to MOSS 2007 we all know there are 3 options to pick from;
In-Place (Quickest, most room for error)
Gradual and (Ease of use and control)
Database (More manual, ease of use, leaves old farm completely in tact)

For all intents and purposes this post will summarize a Database Migration and Resetting Site Definitions, but the focus is primarily on the stsadm command mergecontentdbs and how I used it to restore multiple site collections at once rather than using backup and restore.

Here's what I did:

Backup and Restore (from SPS 2003 to Development MOSS 2007)
  • Put Content DB's in Read Only Mode
  • Run Pre-Upgrade Scan
  • Reset to original Site Definition
  • Backup Content DB's
  • Transfer .bak files to Development
  • Restore DB's in Development SQL Environment

Upgrade and UAT (In Development MOSS 2007)

  • Attach newly upgraded datbase(s) as new content database(s)
  • Upgrade Process (Automatic)
  • User Acceptance Testing

Restore (to Production MOSS 2007)

Once sites have passed UAT and all functionality can be accounted for, sites will be “merged,” or moved to a new content database that is ready to be attached in the production environment. The new database will be backed up and then restored as an entire collection. With this method we can move multiple sites at once rather than manually moving each site collection one by one. The following command line prompt is an example of merging content databases;

First, the option that I will be showing here (#3) requires a file to read from. That file is a list of the sites in the database that you want to move to another one. This list can be edited to exclude certain sites. In our case excluding sites that haven't passed UAT.

Example;

stsadm -o enumsites -url http://sharepoint/ -databasename Content_DB1 > c:\mergesites.txt

Second, perform the mergecontentdbs command.

stsadm -o mergecontentdbs
-url
-sourcedatabasename
-destinationdatabasename
[-operation] {1-3}
1 - Analyze (default)
2 - Full database merge
3 - Read from file
[-filename]

Example;

stsadm -o mergecontentdbs -url http://sharepoint/ -sourcedatabasename Old_DB1 -destinationdatabasename New_DB1 -operation 3 -filename c:\mergesites.txt

The new content database will be restored to the Production SQL Environment and will then be attached to the SharePoint Farm.

  • Restore new merged and backed up database to Production SQL Environment
  • Add New Content Database in Production SharePoint Farm

-Brian Grabowski

Searching for Numbers in SharePoint

If you have been trying to search for a number or a combination of numbers in SharePoint such as an IP address or strictly version numbers, you have probably been coming up short. This is due to "Noise Filtering." There is a text file that filters noise words such as "a," "the," "and," numbers, etc.

Here's what I did:

Edit the noiseenu.txt file
Go to "Sharepoint Portal Server Install directory"/Applications/Guid/Config/noiseenu.txtRemove the "numbers" from the file
Restart the SharePoint Search Service
Reset the search indexes and perform a full crawl

Once the crawl is finished you should now be able to search for numbers.

-Brian Grabowski

Wednesday, September 24, 2008

Adding AD Groups to SharePoint

When adding AD Groups to SharePoint some groups show up while others do not. The case of the matter here depends upon what type of group you are trying to add. Distribution Lists cannot be added only Security Groups can be added.

  • Ensure you are using Security Groups and not Distribution Groups. A Distribution group can not be listed in the DACL and can not be used to assign permissions againstDistribution

  • Lists do not contain the user object, only an "alias"

  • Ensure Security Groups are populated with at least one account

  • Create your Active Directory Groups

  • Add at least one user to each group (and make sure that one user will have their user profile imported) Run a full user profile import in SharePoint Create your audiences (you should now see all the groups that have members with profiles)

-Brian Grabowski

Wednesday, August 13, 2008

The search request was unable to connect to the Search Service

If Search was functioning fine yesterday, but seems to have taken a crap on you today you might want to try these quick and easy fixes followed by some maintenance on Service Accounts and their Logins.

Here's what I did:

Restart these services:

Indexing Service
Office SharePoint Server Search
Windows SharePoint Services Search

After this do a quick IISReset and test the search functionality.

Or heres a quick script to restart all of the services and perform an IISReset for you.

net stop "Indexing Service" && net start "Indexing Service"
net stop "Office SharePoint Server Search" && net start "Office SharePoint Server Search"
net stop "Windows SharePoint Services Search" && net start "Windows SharePoint Services Search"

iisreset \\servername /noforce
iisreset \\servername /status
pause


If this worked for you then you might want to check the Service Accounts that you have tied to these services. They stopped or quit working for a reason. Investigate your Search and Query Server's Event Viewer and see what happened.

-Brian Grabowski

Wednesday, August 6, 2008

SharePoint Latency and Optimiaztion over WAN

Colonel Sanders: Prepare ship for lightspeed!
Dark Helmet: No, no, no! Lightspeed's too slow!
Colonel Sanders: Lightspeed too slow?
Dark Helmet: Yes. We're going to have to go right to.......Ludicrous speed!

One key to an efficient workforce is providing pertinent information in a reliable and quick manner. SharePoint is an excellent foundation for this theory. However, if you have ever wanted to boost speed and increase performace, here is one option that I have used that may give you that extra umph! (THE FOLLOWING REQUIRES REGISTRY CHANGES TO SERVERS...TEST FIRST!!!)

RFC 1323

RFC 1323 is a high-speed transmission path that defines window scaling that allows a receiver to advertise a window size larger than 65,535 bytes.

"A TCP Window Scale option includes a window scaling factor that, when combined with the 16-bit Window field in the TCP header, can increase the receive window size to a maximum of approximately 1GB. The Window Scale option is sent only in synchronize (SYN) segments during the connection establishment process. Both TCP peers can indicate different window scaling factors to use for their receive window sizes. By allowing a sender to send more data on a connection, TCP window scaling allows TCP nodes to better utilize some types of transmission paths with high BDPs."

So in short it allows more information to be sent and recieved, ultimately speeding up performance. In our lab we were able to take an 8MB file that was previously downloading at around 10 seconds to just over 2 seconds.

Pre RFC 1323 tuning= 8MB/10 sec.
Post RFC 1323 tuning= 8MB/2 sec.

Here's what I did:

(The following registry keys may or may not exist...create or edit them...also keep in mind that you can play with the values, however don't increase the sizes so much that they overload the Processor with heavy I/O loads.)

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\AFD\Parameters]

"DefaultSendWindow"=dword:00100a40

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters]

"TcpWindowSize"=dword:00100a40

"GlobalMaxTcpWindowSize"=dword:00100a40

"Tcp1323Opts"=dword:00000003

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HTTP\Parameters]

"MaxBytesPerSend"=dword:000fffff

-Brian Grabowski

Tuesday, August 5, 2008

STSADM, Site Template Codes, and Scripted Site Creation

If ever you've wanted to create Sites from the command line and quit halfway through because you had no clue what the Templates were called or named, you can finally press enter and watch impatiently for the cursor to blink down and say success. Keep in mind these are only the inherent MOSS 2007 Templates and not anything custom that you've thrown in there.

Here's the list:

WSS Templates

Team Site: STS#0
Blank Site: STS#1
Document Workspace: STS#2
Wiki Site: WIKI#0
Blog Site: BLOG#0
Basic Meeting Workspace: MPS#0
Blank Meeting Workspace: MPS#1
Decision Meeting Workspace: MPS#2
Social Meeting Workspace: MPS#3
Multiple Meeting Workspace: MPS#4

MOSS Templates

Document Center: BDR#0
Site Directory: SPSSITE#0
Report Center: SPSREPORTCENTER#0
Search Center with Tabs: SRCHCEN#0
My Site Host: SPSMSITEHOST#0
Search Center: SRCHCENTERLITE#0
Personalisation Site: SPSMSITE#0
Collaboration Portal: SPSPORTAL#0
Publishing Portal: BLANKINTERNETCONTAINER#0
Publishing Site: CMSPUBLISHING#0
Publishing Site with Workflow: BLANKINTERNET#2
News Site: SPSNHOME#0

All you need is the title and the code.

Example;

-sitetemplate STS#0

Also, here is a little code snippet in a batch file that will ask the correct questions, input the data, and create a Site Collection for you without having to go through CA. (Just edit the code to match your configuration.)

@ECHO off
cls
:start
ECHO.
ECHO Please Enter the following information:
set/p url=Complete the Site Url you wish to create. http://mysharepointsite/sites/
set/p owneremail=Enter the Owner's Email address.
set/p ownerlogin=Enter the Owner's Account Name.
set/p sitetemplate=Enter the SiteTemplate Code you wish to use.
set/p title=Enter the Title of the Site.
ECHO 1. Would you like to create the Site Collection?
set choice=
set /p choice=[Y/N] if '%choice%'=='Y' goto CreateSite
if '%choice%'=='N' goto start
ECHO "%choice%" is not valid please try again
ECHO.
goto start
:CreateSite
stsadm.exe -o createsite -url http://mysharepointsite/sites/%url% -owneremail %owneremail% -ownerlogin %ownerlogin% -sitetemplate %sitetemplate% -title %title%
goto end
:end
pause



-Brian Grabowski