Querying Active directory - how to fetch more than 1000 records

I had developed and tested a SSIS package which queried active directory and imported the users into a SQL server table. This was working fine till I deployed this on the client server. Some of the AD users were not getting imported. The users that were not been imported also has the same attributes and belonged to the same OU and domain as the users that were successfully imported. So I got confused and started investigating.

After some hours of various changes to the package and testing, I noticed that only 1000 records were been fetched every time. This surprised me and so I changed the query to fetch only one user which was not getting imported earlier (realised later, that this should have been my first thing I should have tested). This user now got successfully imported. So I concluded that issue was indeed with the records been limited to just 1000.

So now I started searching the internet for a solutions to this wierd problem. There were various solutions where the pagesize could be set when you user directory service objects but I could not find a solution on how to set the pagesize when queried through SSIS. So I started looking whether there is any policy that is set on active directory server and whether it is defaulted to 1000. This is when I found this article
http://support.microsoft.com/default.aspx?scid=kb;en-us;315071&sd=tech

Using the instructions in that article, I did the following
1. At the Ntdsutil.exe command prompt, type LDAP policies, and then press ENTER.
2. At the LDAP policy command prompt, type connections, and then press ENTER.
3. At the server connection command prompt, type connect to server , and then press ENTER.
4. At the server connection command prompt, type q, and then press ENTER to return to the previous menu.
5. At the LDAP policy command prompt, type Show Values, and then press ENTER.

You will see all the values set and can confirm whether the MaxPageSize is set to 1000

This is what Microsoft says about "MaxPageSize"
MaxPageSize - This value controls the maximum number of objects that are returned in a single search result, independent of how large each returned object is. To perform a search where the result might exceed this number of objects, the client must specify the paged search control. This is to group the returned results in groups that are no larger than the MaxPageSize value. To summarize, MaxPageSize controls the number of objects that are returned in a single search result.
Default value: 1,000

We can change this value by running the command
Set MaxPageSize

I am yet to test this on the client's server and will get back whether this helped in solving the problem or not. But this looks promising :-)