Querying Active directory - how to fetch more than 1000 records

I had developed and tested a SSIS package which queried active directory and imported the users into a SQL server table. This was working fine till I deployed this on the client server. Some of the AD users were not getting imported. The users that were not been imported also has the same attributes and belonged to the same OU and domain as the users that were successfully imported. So I got confused and started investigating.

After some hours of various changes to the package and testing, I noticed that only 1000 records were been fetched every time. This surprised me and so I changed the query to fetch only one user which was not getting imported earlier (realised later, that this should have been my first thing I should have tested). This user now got successfully imported. So I concluded that issue was indeed with the records been limited to just 1000.

So now I started searching the internet for a solutions to this wierd problem. There were various solutions where the pagesize could be set when you user directory service objects but I could not find a solution on how to set the pagesize when queried through SSIS. So I started looking whether there is any policy that is set on active directory server and whether it is defaulted to 1000. This is when I found this article
http://support.microsoft.com/default.aspx?scid=kb;en-us;315071&sd=tech

Using the instructions in that article, I did the following
1. At the Ntdsutil.exe command prompt, type LDAP policies, and then press ENTER.
2. At the LDAP policy command prompt, type connections, and then press ENTER.
3. At the server connection command prompt, type connect to server , and then press ENTER.
4. At the server connection command prompt, type q, and then press ENTER to return to the previous menu.
5. At the LDAP policy command prompt, type Show Values, and then press ENTER.

You will see all the values set and can confirm whether the MaxPageSize is set to 1000

This is what Microsoft says about "MaxPageSize"
MaxPageSize - This value controls the maximum number of objects that are returned in a single search result, independent of how large each returned object is. To perform a search where the result might exceed this number of objects, the client must specify the paged search control. This is to group the returned results in groups that are no larger than the MaxPageSize value. To summarize, MaxPageSize controls the number of objects that are returned in a single search result.
Default value: 1,000

We can change this value by running the command
Set MaxPageSize

I am yet to test this on the client's server and will get back whether this helped in solving the problem or not. But this looks promising :-)

WCF - "The client was unable to retrieve service metadata. Make sure the service is running and exposing metadata."

So as you can see, not I have moved onto writing code in framework 3.5 and learning the way around WCF, WPF etc ..

I am in the processes of designing and developing a application using SOA and for that purpose, trying to create the data layer as service. Now why this is been planned has a service has its own reasons and maybe I can explain it sometime later. After creating a simple WCF service, I tried to test it using the default client tool provided by VS2008, I kept getting this error "The client was unable to retrieve service metadata. Make sure the service is running and exposing metadata."

I looked and cross-checked every detail and could not locate the issue. After a couple of trial and error, I noticed that one of the contract defined for the endpoint did not have the complete namespace. I had changed the namespace of contract class and I think I had reassigned the contract using the WCF configuration tool and suprisongly, it had not added the complete namespace. Anyway, correcting this namespace cleared the issue.

So you get a similar error, check the namespace for each of the component within the app.config by opening it within the VS IDE and not within the WCF configuration tool.