Querying Active directory - how to fetch more than 1000 records

I had developed and tested a SSIS package which queried active directory and imported the users into a SQL server table. This was working fine till I deployed this on the client server. Some of the AD users were not getting imported. The users that were not been imported also has the same attributes and belonged to the same OU and domain as the users that were successfully imported. So I got confused and started investigating.

After some hours of various changes to the package and testing, I noticed that only 1000 records were been fetched every time. This surprised me and so I changed the query to fetch only one user which was not getting imported earlier (realised later, that this should have been my first thing I should have tested). This user now got successfully imported. So I concluded that issue was indeed with the records been limited to just 1000.

So now I started searching the internet for a solutions to this wierd problem. There were various solutions where the pagesize could be set when you user directory service objects but I could not find a solution on how to set the pagesize when queried through SSIS. So I started looking whether there is any policy that is set on active directory server and whether it is defaulted to 1000. This is when I found this article
http://support.microsoft.com/default.aspx?scid=kb;en-us;315071&sd=tech

Using the instructions in that article, I did the following
1. At the Ntdsutil.exe command prompt, type LDAP policies, and then press ENTER.
2. At the LDAP policy command prompt, type connections, and then press ENTER.
3. At the server connection command prompt, type connect to server , and then press ENTER.
4. At the server connection command prompt, type q, and then press ENTER to return to the previous menu.
5. At the LDAP policy command prompt, type Show Values, and then press ENTER.

You will see all the values set and can confirm whether the MaxPageSize is set to 1000

This is what Microsoft says about "MaxPageSize"
MaxPageSize - This value controls the maximum number of objects that are returned in a single search result, independent of how large each returned object is. To perform a search where the result might exceed this number of objects, the client must specify the paged search control. This is to group the returned results in groups that are no larger than the MaxPageSize value. To summarize, MaxPageSize controls the number of objects that are returned in a single search result.
Default value: 1,000

We can change this value by running the command
Set MaxPageSize

I am yet to test this on the client's server and will get back whether this helped in solving the problem or not. But this looks promising :-)

WCF - "The client was unable to retrieve service metadata. Make sure the service is running and exposing metadata."

So as you can see, not I have moved onto writing code in framework 3.5 and learning the way around WCF, WPF etc ..

I am in the processes of designing and developing a application using SOA and for that purpose, trying to create the data layer as service. Now why this is been planned has a service has its own reasons and maybe I can explain it sometime later. After creating a simple WCF service, I tried to test it using the default client tool provided by VS2008, I kept getting this error "The client was unable to retrieve service metadata. Make sure the service is running and exposing metadata."

I looked and cross-checked every detail and could not locate the issue. After a couple of trial and error, I noticed that one of the contract defined for the endpoint did not have the complete namespace. I had changed the namespace of contract class and I think I had reassigned the contract using the WCF configuration tool and suprisongly, it had not added the complete namespace. Anyway, correcting this namespace cleared the issue.

So you get a similar error, check the namespace for each of the component within the app.config by opening it within the VS IDE and not within the WCF configuration tool.

Sharepoint, MOSS custom project/solution

I had some work recently in creating a custom solution for sharepoint and I had never worked in sharepoint before. I thought just like you find a lot of solutions and books of other technologies, I would be able to find similar things for sharepoint too .. But sadly that was not the case. There are lots of artcles, books on sharepoint administration/customization, but very less on actual development. You will find examples for creating web parts, adding custom lists etc. But my requirement has something more than all this

I had to create a package which could be given to a client, who could then install it into his sharepoint server. Then my new sharepoint "site" must be added as a "site" within sharepoint and the users must be able to see a customzed solution which queries my database and does various other things ..

This involved ..
- creating a solution
- creating a package
- adding custom web parts
- adding these custom web parts to the "default.aspx" page
- installing this package as a "solution" in sharepoint
- having this installed soltuon as a "template" in the template list while creating new "site"
- after creating this new site, the site must have my setting for the default.aspx page and follow my styles defined

How did I do all of these. Hopefully I get time to write it soon .. If there is anyone out there who needs a answer right now, do write to me .. or I will hopefully post all that I did in the next coupld of weeks

I am not sure whether the approach I took was the best one, but at least it worked for me ..

Rejo

SSIS - debug a "script component" control

If you are reading this, you might also have faced a situation where you added a breakpoint and thought of debugging a script component in SSIS package ..

Ok, I found this is not possible. At least I could not do it in SQL 2005. Maybe its possible in 2008. Do note that you can debug scripts in a "Script task" control but not within a "script component" control within a data flow section.

I tried everything .. changing the "Precompile" property to false/true, changing the script control's name (in the design script window - changed the default generated control name from ScriptComponent_7dac1823375d4ad0bddf242bed9e1391 to ScriptComponent) etc .. but nothing worked ..

So maybe this post will save someone's time .. I copied my code to script task and I could manage to debug part of the code (the part I needed to check) .. after that I deleted the script task and returned back to my original flow/code of using the "script component"

Rejo

System.Object(), SSIS, Active directory and MemeberOf property

For the last couple of months I have working on sharepoint solutions and one of the requirement involved in fetching all users and groups from active directory. The data fetched had to be then inserted into a local SQL databaseb. As this information changes with time, the process had to be run once daily. So I opted on using SSIS to automate the whole process

It seemed pretty straight forward. I used a OLEDB connection of "Microsoft data services" to read the data from active directory. As a test I fetched couple of fields like the cn, name etc and everything worked fine, till I decieded to get the groups that the users belong to. I knew that it would be part of the "memberof" property. I used the same approach and I noticed that value gets inserted as "System.Object[]" into the database. I figured out that the value is returned as a blobcolumn and anyway I tried to parse it, it always gave me the same value. I tried

Dim lByte() As Byte = CType(dataRow(0), Byte()) .. where datarow(0) is the memberof field
System.Text.Encoding.Unicode.GetChars(lByte)

... also tried converting the column to string, collection, byte etc etc ..

and many other options .. all gave me the same result or an error .. I was really getting frustrated and I searched the net for solution. There are some examples on using SSIS and active directory and no where do they talk about how to read value from this column. So I came back to trying it on the own. I wanted to check the type of this column. But I could not add a breakpoint into a "script component" (and that is another story) .. I learned that I could add a breakpoint into a "script task" and so I copied my code over there and added breakpoint to see what data is fetched.

I saw that the column actually contains a array value. I had never seen this before .. The content of a single column looked like
(0) CN=CN1,OU=OU1,DC=corp1,DC=corp2
(1) CN=CN2,OU=OU1,DC=corp1,DC=corp2
(2) CN=CN3,OU=OU1,DC=corp1,DC=corp2

so i converted the column value to "system.array". and got the information that I needed ...

Hope this helps some people out there .. and not have to spend hours or days looking for a solution.

Rejo

Mapping varchar(max) - using micorosoft application data blocks (No mapping exists from DbType .. to a known SqlDbType ....)

I have been happily using application blocks for various tasks in my project and the data blocks was one of them. I had code like this which worked fine ..

Dim lDatabase As Database = DatabaseFactory.CreateDatabase()
Dim lCommand As DbCommand = lDatabase.GetStoredProcCommand("dbo.pr_service_WriteLog")
lCommand.CommandType = CommandType.StoredProcedure
'-- add the parameters
lDatabase.AddInParameter(lCommand, "@CreatedOn", SqlDbType.DateTime, DateTime.Now)
'-- add the error type
lDatabase.AddInParameter(lCommand, "@LogType", SqlDbType.VarChar, EntryType.ToString)
lDatabase.AddInParameter(lCommand, "@LogType", SqlDbType.VarChar, lLogBase.Type.ToString)
'-- add the message
lDatabase.AddInParameter(lCommand, "@LogMessage", SqlDbType.VarChar, lLogBase.Message)

lDatabase.ExecuteNonQuery(lCommand)

This worked fine for quite sometime till I had a new requirement to write a SP which had to insert into a table a fields with the data type defined as varchar(max). To apss this new type from the UI to SP I tried
lDatabase.AddInParameter(lCommand, "@NewParam", SqlDbType.text, MyValue)

It gave the error "No mapping exists from DbType UInt16 to a known SqlDbType." ..
So I tried various other types like Nvarchar etc etc .. but all failed .. Then I started reading about the data types supported by the data block and I came to realise that instead of using SQLDbType, i could use DBtype directly. So I tried DBtype.string and it worked !!!

Hope this helps someone who might get stuck in a similar area ....

"Exception has been thrown by target of invocation" Error

You might face this error and wonder what happened. I had a similar situation today. The code was working fine and all of a sudden I got this error. I am not that kind of person who beleives very strongly that all errors are due to some bad coding, maybe by the developer or maybe thrown bt the tools used for development. Till now, whenever I have been involved in coding or debuging someone's problem , I have managed to at least find out where the issues have been and most of time solve it too. Anyway, so I started tracing my steps backwards and commenting recenlty changed codes. But still was not able to find exactly where the error is occuring.

Finally, during one my debug attempts, I happened to look at the exception object's inner exception and noticed the error was because of a new addition I had on the app.config file. I made the correction and the error was gone. BTW, the change I did was add a "configSections" tag at the end of the app.config file. If you add a "configSections" tag, it has be the first line after the "configuration" tag.

So to conclude, if you get this error ""Exception has been thrown by target of invocation"", look at the inner exceptions. Keep looking recursively into the inner exception's "inner exception" till you see "nothing" and hopefully you see the actual error message and correct the problem.

happy coding !!

Public member 'OverRideMethod' on type 'ObjectHandle' not found. [Reflection - Access methods from class dynamically]

I had a hard time with this. Spend around 2-3 hours trying various options till i got a clue from one of the links on the internet.

I had a class library (dll) referenced in a windows project and all the classes in this dll is inherited from a base class. The baseclass is mustinherit and has mustoverride function. My plan was that this windows application would have a list of (dynamically maintained using a XML file in the windows application) classes and its methods that can be executed from this dll. Each of the class within this dll overrides the mustoverride function from the baseclass and does some actions. The reason for this design was to make a "plug and play" kind of application. So if i wanted to add a new functionality, i just need to add a new class to the dll, edit the XML file and the front-end will show this new functionality and it will execute it. There were many other functions that were specific to the project that need not be discussed here .. Anyway, this sounded very simple to me and i started with writing the code..

The first thing that came into my mind was whether i would have to use reflection. But the dll was referenced in the main windows project. So there was no need to load a assembly dynamically and i did not want to go with that approach. Then found from the help about Activator.CreateInstance which has a overloaded function which accepts 2 strings .. the assembly name and the class name .. this was what i wanted. So i wrote this

Dim lObject As Object
lObject = Activator.CreateInstance("Namespace.MyAssembly", "Derived Class Name")
lObject.OverRideMethod

But this kept giving an error on the line where it is executing overriden method.
Public member 'OverRideMethod' on type 'ObjectHandle' not found.

I was sure that the object was getting created because when i changed the assembly or the class name, it gave the error on that line. So what was the issue? I could not get any help in MSDN. So i started searching the net and luckily got a link which said that Activator.CreateInstance does not give a object in return, but returns a object handle. I checked MSDN and it said that it gave back a object. I aklso noticed now that the error I got mentioned that "... on type ObjectHandle not found" So I decided to give it a try.. So I changed the code to

Dim lHandle As System.Runtime.Remoting.ObjectHandle
lHandle = Activator.CreateInstance("Namespace.MyAssembly", "Derived Class Name")
lRulesObject = lHandle.Unwrap()
lRulesObject.OverRideMethod

Lo and behold, it worked !! ..

I also type cast my object to the baseclass so that I can now call the method from the baseclass directly.. So my final code was

Dim lHandle As System.Runtime.Remoting.ObjectHandle
Dim lRulesBase As MyBaseClass
Dim lRulesObject As Object

lHandle = Activator.CreateInstance("Namespace.MyAssembly", "Derived Class Name")
lRulesObject = lHandle.Unwrap()
'-- cast it to the base class
lRulesBase = DirectCast(lRulesObject, MyBaseClass)
'-- call the service method
lRulesBase.OverRideMethod()

I spend some time finding this out and i thought I will post this for others who might have similar issues.

Happy coding !!

Icon size in notifyicon

Recently I had a requirement to create a windows service in .net. This service also had to have a "form" like a "dashboard" functionality that showed the status of the service and various options that the user could do to make changes to the service functions. For this, I decided to show a icon in the taskbar when the service starts and onclick of this icon, I would show the form.

This can be done using notifyicon control in .net and its pretty straightforward. But the issue I was facing was with the icon been shown in the taskbar. It was never clear and after trying various options like different size etc, i searched the net and found this article which was useful - http://www.hhhh.org/cloister/csharp/icons/

------ from the above link ------
"A reader asked for a quick-reference table at the top of this document, showing the default icon versions used by Windows Forms applications in different circumstances, so here it is. Detailed explanations are provided below if you're interested

Circumstance: Icon Version:
Control Box 16x16x256
Taskbar 16x16x256
Notify Icon 32x32x256, shrunk down (see below)
Desktop Shortcut 32x32x256 (Normal sized icons)
48x48x256 (Large sized icons)
Add/Remove Programs 16x16x256
Folder Browse dialog 16x16x256

Notify IconIf you add a NotifyIcon control to your application's Form object, then your application will display an icon in the tray area of the taskbar. The rendered size of the notify icon is the same as for the Control Box and Taskbar icons: 16x16 pixels, potentially stretched to accomodate the height of the system tray, which is dependent on the height of the Taskbar. Oddly, however, Windows does not use the 16x16x256 bitmap for this circumstance; it uses the 32x32x256 bitmap by default, and squashes it down to the required size. I can think of no particularly good reason for this difference--using the 32x32 version for the notify icon, but 16x16 for taskbar and control box--but there it is.

16x16 Notify Icon 18x18 Notify Icon


Notice that the text on my 32x32x256 icon is barely readable when shrunk down to 16x16 size. The same will be true for any one-pixel-wide details you include in your 32x32x256 bitmap, unless you are very careful about where you put them. The reason for this is clear: a 16x16 bitmap has only one-quarter as many pixels as a 32x32 bitmap. Consequently, three-quarters of the information in the 32x32 bitmap is thrown away when shrinking it down to 16x16. What to do about it is less clear.

To get a more optimal appearance for your NotifyIcon, you need to know which pixels out of the 32x32 bitmap are the ones that will be displayed at 16x16 size. Fortunately, the mapping is straightforward. Windows throws out all the even-numbered rows and columns, and keeps what's left. Shown here at double-size is a 32x32x256 bitmap, where the black pixels are the ones that will be seen when the bitmap is shrunk down to 16x16 on a NotifyIcon, and the actual NotifyIcon image that this bitmap turns into in the system tray:
Before: ...and after:
As you can see in these two images, the black pixels in the 32x32 image are the ones that are actually displayed at 16x16 size. The yellow has disappeared because the even-numbered rows and columns are thrown away. Pixels in the even-numbered rows and columns are important for the appearance of desktop shortcut icons, but are irrelevant for NotifyIcon objects.

Custom validator controls : Change style of label when error occurs

ASP.net

We had this unique requirement from a client that when a validation failure occurs like "required field is not filled", the corresponding label must be highlighted with a different color. As all of you might already be aware, we attach the validator controls to the control that needs to be validated and not the label. For eg, if you have a "Customer name" field with a textbox (txtCustomer) which is mandatory, you add a requiredFieldValidator with its ControlToValidate property = txtCustomer.

So how do you now highlight the label "Customer name" .. lets say with a different background color. Ofcourse, doing that on the server is easy, but that negates the reason of using validator controls which does everything on the client (browser). So I decided to build a custom control by inheriting from the standard requriedFieldValidator control. I added a new property called LabelToHighlight which will accept a control name whose styles needs to be changed when an error occurs. I reaserched on how to allow user (consumer the control) to be able to select a label (like you selecting controls when setting the ControlToValidate property) and not just type in a control name and I came across typeconvertor settings available for properties .. anyway, given below is some of the code .. you can write to me for more explanation if needed ..

'Created by Rejo on Aug 23rd 2007
' Standard field validators has just the controltovalidate property which can be set to specify which
' controls needs to be validated. But the delivery.com project requires the errors to highlight the label
' of the control which is been validated.
' This custom validator does just that.
' A new property added gives option to the developer to select a control which gets highlighted with a different style.
' The styles are set as embeded resource and should be changed as per the requirement of the project
' The styles too could have been exposed as public properties, but that would be each control will have to manually
' set by the developer.
' The javascript which changes the style is also embeded into the project
' ONLY THE DLL NEEDS TO BE REFERENCED IN THE MAIN PROJECT which will consume these controls

Imports Microsoft.VisualBasic
Imports System.Web.UI
Imports System.ComponentModel
Imports System.Web.UI.WebControls

Namespace Controls

_
Public Class CustomRequiredValidator
Inherits System.Web.UI.WebControls.RequiredFieldValidator
Dim lstrLabelToHighlight As String

_
Public Property LabelToHighlight() As String
'property to hold the name of the label control which would be highlighted when the error occurs
Get
Return lstrLabelToHighlight
End Get
Set(ByVal value As String)
lstrLabelToHighlight = value
End Set
End Property

Protected Overrides Sub AddAttributesToRender(ByVal writer As System.Web.UI.HtmlTextWriter)
'set the display property to none
Me.Display = ValidatorDisplay.None
MyBase.AddAttributesToRender(writer)
'add the custom properties
writer.AddAttribute("LabelToHighlight", Me.LabelToHighlight)
End Sub

Private Sub CustomRequiredValidator_PreRender(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.PreRender
Dim ctlParent As System.Web.UI.Control = Me.Parent
Dim lLableHighlight As Label = TryCast(ctlParent.FindControl(LabelToHighlight), Label)

While lLableHighlight Is Nothing
ctlParent = ctlParent.Parent
If ctlParent IsNot Nothing Then
lLableHighlight = TryCast(ctlParent.FindControl(LabelToHighlight), Label)
Else
Throw New Exception("Control to highlight is not found.")
End If
End While

LabelToHighlight = lLableHighlight.ClientID

'check if the script is already present and if not, then add it
If Not Me.Page.ClientScript.IsClientScriptIncludeRegistered("ControlsCommon.js") Then
Dim lstrScriptLocation As String = Page.ClientScript.GetWebResourceUrl(Me.GetType(), "Delivery.ControlsCommon.js")
Page.ClientScript.RegisterClientScriptInclude("ControlsCommon.js", lstrScriptLocation)

' add the style .. assumming that the style too would be added only once along with the js file

Dim lstrLocation As String = Page.ClientScript.GetWebResourceUrl(Me.GetType(), "Delivery.ControlStyle.css")
Dim lobjIncludeLiteral As LiteralControl = New LiteralControl(String.Format(lstrTemplate, lstrLocation))
CType(Page.Header, HtmlControls.HtmlHead).Controls.Add(lobjIncludeLiteral)

'register submit script which will call the script which will highlight the control if there is a error
Me.Page.ClientScript.RegisterOnSubmitStatement(Me.GetType(), "HighlightLabelSubmit", "UpdateErrorLabel();")
End If
End Sub
End Class
End Namespace

Code from the COMMON.js file

Function UpdateErrorLabel()
{
var lobjValidator;
//first reset all the existing validators to default style
for (i = 0; i < Page_Validators.length; i++)
{
lobjValidator = document.getElementById(document.getElementById(Page_Validators[i].id).getAttribute('LabelToHighlight'));
if (lobjValidator != null)
{
lobjValidator.className = 'lblnormal';
}
}


//change the style for all error labels
for (i = 0; i < Page_Validators.length; i++)
{
if (!Page_Validators[i].isvalid)
{
//lobjValidator = document.getElementById(Page_Validators[i].LabelToHighlight);
lobjValidator = document.getElementById(document.getElementById(Page_Validators[i].id).getAttribute('LabelToHighlight'));
if (lobjValidator != null)
{
//lobjValidator.setAttribute('class', 'lblerror');
//lobjValidator.setAttribute('className', 'lblerror');
lobjValidator.className = 'lblerror';
}
}
}
}

This does have some flaws which I have learnt to overcome like sometimes when you add the this control, the highlight does not work. As you can see from the code, I have to add on to the standard validation code that asp.net generates which checks for the isvalid property and then change the style when it is false. Some my functions get added to the front and so this js always finds the isvalid = true and does not validate. For these cases, I just remove the Custimvalidator and add it again and it works .. :-) ..

Note, you will need the style 'lblError' etc to be available in your stylesheet referred by the page ..

This line is not getting accepted by this blogpost when added in my code above..

it must be in the prerender event .. i will try and make some changes to the code so that this post gets accepted
Dim lstrTemplate As String = "<link rel='stylesheet' text='text/css' href='{0}' CLOSING TAGS"