Membership & BDC import status hangs with infinite items (SharePoint 2007)

This was a bit of an odd one I came across, caused by a transaction log filling up. Take the following scenario

1. The log file for SharePoint_SSP_Search (the SSP Search database) becomes full, and cannot grow further. The database stops responding
2. The Office Search Service , then the SharePoint timer service throws an error as it cannot contact the database. This causes an error when the crawler tries to pause, and the object cache becomes corrupted (slight speculation here).
3. The SharePoint services continually attempt to communicate with the SQL Server (every second), the net result being that the server is too busy to serve RDP requests (not ideal if doing remote support!)
4. Someone “resolves” the database log size, and restarts all the SharePoint services, including IIS, the Timer Service and Office Search (or server reset)
5. Office Search and Timer resumes, persisting the corrupted object cache data to the database. This has a knock on impact on the “Membership and BDC Import”, which gets stuck in some form of infinite loop (last count, over “56,000” AD records had been imported, in an AD of less than 2,000!)
6. This fills up the transaction log at the rate of a few 100MB a minute, quickly reaching the max log file size again, causing SharePoint to stop responding once more.

The route cause of all this was that the transaction log for the search database was full. In this scenario the reasons was:

1. The log file had been capped at 5GB. This is not necessarily unreasonable, but if it does reach the limit, this triggers the issue.
2. The database is in simple recovery mode
3. The log file will grow with each SharePoint content crawl, or user profile import.
4. The database backup for SharePoint_SSP_Search appears to not be on a daily schedule (last backup was more than 10 days previous). This means that the log file has significantly more time to grow, and shrinking the log nightly has no impact, due to the fact that the database (despite being in simple mode recovery) has not been backed up, and the transaction log space has not been freed for deletion.

To resolve the problem, you can take the following steps:

1. Manually backup/shrink the search database. Note, if your database is maxed on the log file, you may need to allocate more log file space before you can succeed in a manual backup/shrink
2. Created a particular maintenance plan for this database, which backs up nightly, then shrinks the log file. (this stops the trigger for the issue from occurring under "normal" conditions.
3. Stop the troublesome “Membership and BDC Import”
4. Reset all crawled content, which clears out the corrupted data from the search database
5. Started a full user profile import and confirm success.
6. Initiated a full crawl on “Local Office SharePoint Server Sites”, and confirm success

I hope this helps someone else out. If you find a more definitive route cause or fix, please let me know.