Sunday, 22 January 2012

C# with Visual Studio and Rational Team Concert (OSLC revisted) (Part 1)


I have had some recent customer experiences which have impelled me to write some blogs on the RTC Visual Studio integration. This topic follows the previous series well, where I wrote about RTC and CDT. In fact, I will will basically recreate the OSLC example ( this time using C# ) but using the Visual Studio 2010 development environment instead. Meaning that I will be using visual studio not only for developing but writing unit tests and doing a static analysis all while using the RTC visual studio integration.

If you recall in the previous blog series on OSLC, I created an OSLC client in C/C++ using CDT. I could do the same in Visual Studio (OSLC client using C/C++, that is) but I will instead use C# to create the OSLC client this time to show how this can also be done.

What You Need

This exercise may work with other versions of Visual Studio as the integration supports other versions as well but I will use Visual Studio 2010 .

I am also assuming that these are installed as well as:


Ensure that the CLM 3.0.1.2 servers are setup correctly (at a minimum, CCM and JTS). Please refer to the CLM 3.0.1.2 information center for how to do this. For reference my URLs and directories area as follows, you will need to replace them in this blog as appropriate for your environment:

JTS URL: https://tpborisk7:9443/jts
CCM URL: https://tpborisk7:9443/ccm
CLM install directory: C:\IBM\CLM3012
Sandbox: c:\vsworkspace
Setting up (creating project areas and solutions)

1. If you have installed your visual studio environment correctly, you can launch Visual Studio now. The Visual Studio RTC integration behaves very much like the Eclipse RTC integration. There are some key differences.

2. One of the differences is that it is not possible to create a CCM project area from visual studio. This will need to be done through the CCM web UI. Go to the project area creation page: https://tpborisk7:9443/ccm/admin#action=com.ibm.team.process.editProjectArea&itemId=new

3. Create a new project area called OSLC and use the scrum template. Be sure to assign the user (in my case servadm) you will be connecting from Visual Studio as the “Scrum Master”.

4. Now connect to the project area using the Team Artifacts window:
5. Connect to the server:

6. Click next and select the OSLC project area we created and click "Finish". We are now connected.
(NOTE: You may need to "Load.." your repository workspace into your sandbox at this point in order to see the "Share solution in Jazz" option below.)

7. Lets create a new solution that we will use to develop our c# OSLC client.In Visual Studio select
File->New->Project...

8. Select a Console application under Visual C#->Windows->Console Application

9. Call the project OSLC Consumer and use a solution called OSLC (as we will created more projects in the future) in a preferred location. Make sure to select the "Share solution in Jazz" option.


10. Once you confirm your solution location you will be asked to select a sandbox. You should choose the sandbox location that we chose as the solution directory above. You can think of a sandbox as a collection of visual studio solutions/components under Jazz source control. So:


11. Lets create a new component for our Consumer called “OSLC”

12. Leave the defaults for the ignore files options. You should end up with the following directory structure (or similar):


13. In your pending changes view you should now see that we could deliver the new solutions with its project into jazz source control which we will do. Right click on the "Share Projects" and "Deliver" them into the OSLC Stream. (Click yes if it asks you if you want to deliver the component changes)


14. (If you understand component solutions relationships in VS skip to 17)
In anticipation of unit test we will eventually create lets create two more project skeletons. One called "OSLC Consumer Common"(containing common code) and a unit test project, "Test OSLC Consumer". Right click on the project and Add a project of type "Class Library"

15. Let's add the test project called "Test OSLC Consumer" of type "Test Project":


16. Unlike the solution there is not default option to add a project to Jazz Source control so we will have to do it now. Select both the new projects and right click and select the “Share Project(s) in Jazz...” option but do not click Finish!


NOTE!!
Notice a key difference here between the RTC eclipse and Visual studio integration. The Component alignment is done at the solution level rather then at the project level! 
So in eclipse many projects are associated to a component, where in visual studio, many solutions to a component, where projects in the solution belong to the component of a solution.
In effect, if we wanted to have the unit tests, common code and consumer client in different components you would essentially need to create three solutions (one for each) in the sandbox. As this is exactly what we want this is what we will do (but you don't have to).
Just because we have three solutions does not mean we need to keep three IDE solution instances open because we can easily add the projects from different solutions to the same solution (which we will do.)
As I have gone through this detour to highlight this difference, lets revert and cancel out of the “Share Projects(s) in Jazz...”.. Right click on the two projects and select Right Click->remove. Also delete the corresponding directories on the file system. You will probably also want to delete the following files out of the solution directory: Local.testsettings, OSLC.vsmdi and TraceAndTestImpact,testsettings.

17. In anticipation of unit test we will eventually create lets create two more solution skeletons. One called "OSLC Common" with a project called "OSLC Consumer Common" (containing common code) and one called "Test OSLC" with a unit test project, "Test OSLC Consumer". As we want these in separate components lets create two solutions for these projects. We will create these solutions in the same sandbox as the "OSLC" Solution above.
18. First create the "OSLC Consumer Common" library which is a project of type "Class Library" in the "OSLC Common" solution:


19. Put it in the "OSLC Common" Component:


20. Delver the new solution and project to the OSLC Stream.

21. Let's repeat for "Test OSLC Consumer" library which is a project of type "Test Project" in the "Test OSLC" solution:


22. Now lets add it to the "Test OSLC" component:


23. Deliver the solution to the “OSLC Stream”

24. Go to the Solution Explorer Window (Team Concert->Windows->Sandbox Explorer).. it should looks as follows:


25. As i have named solutions the same as the Components we can easily see the projects in each component.
To avoid having to keep three IDEs open lets add the "OSLC Consumer Common" and "Test OSLC Consumer" projects to the "OSLC" solution by right clicking on the solution and Add->Existing Project...


26. Add the two projects and save the solution. You should have:

27. You can now deliver the OSLC.sln file to keep the changes to the solution. In the future we only need to open the OSLC solution. When you switch sandboxes Visual Studio will automatically ask you for a solution file. We will pick OSLC.sln from now on.


28. I have also gone ahead an deleted the default component and we might as well have the components be owned by the project level. Go to the “Team Artifacts” window and right click on the components and change ownership to the project for each component:


You project areas should look like this now:

This should prepare us for the next blog entry where we will start coding the OSLC client in C#.

17 comments:

  1. Hi Boris,

    Thanks for the thoughtful example.

    I did install Jazz Team server long with Doors NG, CCM tools, etc. The example COSLCConsumer runs fine to get the ccm root services. However, it seems there are issues with the returned xml, as the it will throw exceptions at the call:

    XPathDocument doc = new XPathDocument(response.GetResponseStream());

    I tried to check the returned XML document, it inform the XML have the wrong syntax.

    If you could provide me any hints to overcome this, it would be great!

    Best regards,

    ReplyDelete
  2. Did you look at the XML using a browser? Does it look ok?
    Make sure its ok in the browser, perhaps it returned using a different content type such as JSON.

    ReplyDelete
  3. Thanks for the reply.

    Yes, it is XML, I did check the syntax of the returned XML using a browser, and there are just few syntax errors, if I manually change it, the syntax will be OK.

    If I just enter the link of root services in a browser, like:
    https://localhost:9443/rm/rootservices
    I can get a return XML with the correct syntax, however when I call the children links with GET methods for REST web services using your code, an exception will be raised.

    My assumption is that the Jazz team server is still in Beta, and not finish implementation yet, so we will need to wait? is that right? Also I could not find the details document of the web services API we can call for the apps such as DOORS NG on Jazz server! If you could share me some info in regarding to these issues, it would be great!

    Thank you so much for looking into this.

    ReplyDelete
    Replies
    1. Yes, if there are syntax errors it could very well be a defect with the beta. Can you post the XML and I can open up a defect for you if you wish. Or you can submit one your self at https://jazz.net/development/bugs/ and subscribe me to the work item (@kuschel)

      Delete
  4. Thanks,

    I have created an item: Defect 56376

    Here is the link:
    https://jazz.net/jazz03/web/projects/Requirements%20Management#action=com.ibm.team.workitem.viewWorkItem&id=56376

    ReplyDelete
  5. Hi Boris,

    The code now runs fine with CCM... however, when I change it to rm link, it will fail! for example:
    string server = "https://192.168.1.167:9443/rm";
    string url = "https://192.168.1.167:9443/rm/rootservices";

    .....

    manager.AddNamespace("oslc_rm", "http://open-services.net/xmlns/rm/1.0/");

    XPathNodeIterator iterator = nav.Select("/rdf:Description/oslc_rm:rmServiceProviders/@rdf:resource", manager);

    Do you have any ideas how rm (requirement management) is diffrence with CCM?

    Thanks,

    ReplyDelete
  6. Hi,

    The code can get the rmServiceProviders fine (the catalog access point
    https://192.168.1.167:9443/rm/discovery/RMCatalog).

    However, on next access call to get the Project areas, it will throw an exception -unauthorised access...because it is the secured page.

    It seems that CCM is using form based form-based authentication to access to the Service Providers, while RM is using OAuth authentication mechanism to access the Service Providers, so the code does not work with RM.

    The quesiton is, how CCM and RM can be difference in authentication, while they have lots of similarities in Jazz server.

    If you could help to get the code work for RM, it would be great!

    Thanks,

    ReplyDelete
    Replies
    1. Ah, I see what you mean. Ok, yes, you will have to authenticate with JTS as well. I modified the authentication routine. I plan to write a more comprehensive OSLC example where i will include something like this. I also found a strange bug in my installation. Not sure if you get this as well. You will need to change the passed in serverURI as well to be without context as now it is searched for.

      Delete
  7. const String OAUTHREQUIRED = "X-jazz-web-oauth-url";
    const String SETCOOKIE = "Set-Cookie";
    public static HttpWebResponse sendGetForSecureDocument(string serverURI, HttpWebRequest request, string login, string password)
    {
    // Step (1): Request the protected resource
    HttpWebRequest documentGet2 = (HttpWebRequest)WebRequest.Create(request.RequestUri);
    documentGet2.Method = request.Method;
    documentGet2.CookieContainer = request.CookieContainer;
    documentGet2.Accept = request.Accept;
    if (request.Headers.Count > 0)
    {
    foreach (string key in request.Headers.Keys)
    try
    {
    documentGet2.Headers.Set(key, request.Headers[key]);
    }
    catch (ArgumentException)
    {
    }

    }
    documentGet2.Timeout = request.Timeout;

    if (DEBUG) Console.WriteLine(">> GET(1) " + request.RequestUri);

    HttpWebResponse documentResponse = null;
    try
    {
    documentResponse = (HttpWebResponse)request.GetResponse();
    if (DEBUG)
    {
    Console.WriteLine(">> Response Headers:");
    HttpUtils.printResponseHeaders(documentResponse);
    }
    }
    catch (WebException e)
    {
    if (DEBUG)
    {
    Console.WriteLine(">> Response Headers:");
    HttpUtils.printResponseHeaders((HttpWebResponse)e.Response);
    }
    string theader = e.Response.Headers[OAUTHREQUIRED];
    if ((theader != null))
    {
    HttpWebRequest formPost = (HttpWebRequest)WebRequest.Create(theader);
    formPost.Method = "POST";
    formPost.Timeout = 30000;
    formPost.CookieContainer = request.CookieContainer;
    formPost.ContentType = "application/x-www-form-urlencoded";
    String output = "authorize=true";
    Byte[] outputBuffer = Encoding.UTF8.GetBytes(output);
    formPost.ContentLength = outputBuffer.Length;
    Stream stream = formPost.GetRequestStream();
    stream.Write(outputBuffer, 0, outputBuffer.Length);
    stream.Close();
    // Step (2): The client submits the login form
    if (DEBUG) Console.WriteLine(">> POST " + formPost.RequestUri);
    documentResponse = (HttpWebResponse)formPost.GetResponse();
    if (DEBUG) HttpUtils.printResponseHeaders(documentResponse);
    }
    else throw e;
    }
    if (documentResponse.StatusCode == HttpStatusCode.OK)
    {
    string header = documentResponse.Headers[AUTHREQUIRED];
    string authcontext = null;
    if ((header != null) && header.Equals("authrequired"))
    {
    string contextheader = documentResponse.Headers[SETCOOKIE];
    authcontext = contextheader.Split(";".ToCharArray(), 2)[1].Split("=".ToCharArray(), 2)[1];
    documentResponse.GetResponseStream().Flush();
    documentResponse.Close();
    // The server requires an authentication: Create the login form

    //BUG: Some bizzare bad request behaviour. Hit the auth URL once cleans it
    HttpWebRequest formPost = (HttpWebRequest)WebRequest.Create(serverURI + authcontext + "/j_security_check");

    try
    {
    formPost.Method = "POST";
    formPost.Timeout = 30000;
    formPost.CookieContainer = request.CookieContainer;
    formPost.ContentType = "application/x-www-form-urlencoded";
    String eoutput = "j_username=" + login + "&j_password=" + password;
    Byte[] eoutputBuffer = Encoding.UTF8.GetBytes(eoutput);
    formPost.ContentLength = eoutputBuffer.Length;
    Stream estream = formPost.GetRequestStream();
    estream.Write(eoutputBuffer, 0, eoutputBuffer.Length);
    estream.Close();
    documentResponse = (HttpWebResponse)formPost.GetResponse();
    }

    ReplyDelete
  8. catch(Exception)
    {
    if(documentResponse != null)
    {
    try
    {
    documentResponse.GetResponseStream().Flush();
    documentResponse.Close();
    }
    catch (Exception)
    {
    }
    }
    }
    //END BUG

    formPost = (HttpWebRequest)WebRequest.Create(serverURI + authcontext + "/j_security_check");
    formPost.Method = "POST";
    formPost.Timeout = 30000;
    formPost.CookieContainer = request.CookieContainer;
    formPost.ContentType = "application/x-www-form-urlencoded";
    String output = "j_username=" + login + "&j_password=" + password;
    Byte[] outputBuffer = Encoding.UTF8.GetBytes(output);
    formPost.ContentLength = outputBuffer.Length;
    Stream stream = formPost.GetRequestStream();
    stream.Write(outputBuffer, 0, outputBuffer.Length);
    stream.Close();

    // Step (2): The client submits the login form
    if (DEBUG) Console.WriteLine(">> POST " + formPost.RequestUri);
    HttpWebResponse formResponse = formResponse = (HttpWebResponse)formPost.GetResponse();
    if (DEBUG) HttpUtils.printResponseHeaders(formResponse);

    header = formResponse.Headers[AUTHREQUIRED];
    if ((header != null) && header.Equals("authfailed"))
    {
    // The login failed
    throw new WebException("Authentication failed");
    }
    else
    {
    formResponse.GetResponseStream().Flush();
    formResponse.Close();
    // The login succeed
    // Step (3): Request again the protected resource
    if (DEBUG) Console.WriteLine(">> GET(2) " + request.RequestUri);
    return (HttpWebResponse)documentGet2.GetResponse();
    }
    }
    }
    return documentResponse;
    }

    ReplyDelete
  9. Thanks, it works!

    Just a note, the code:
    formPost = (HttpWebRequest)WebRequest.Create(serverURI + authcontext + "/j_security_check");
    seem cause the invalid url as:
    ....jts_server_ip:9443/rm/jts/j_security_check

    while correct path should be:
    jts_server_ip:9443/jts/j_security_check

    Just change it to be:
    formPost = (HttpWebRequest)WebRequest.Create(jtsUrl + "/j_security_check");
    (jtsUrl = "https://192.168.1.167:9443/jts")
    and the code run fine for me.

    If you can give a complete example with code changes that works for both RM and CCM, it will be great. An interesting adding function such as getting requirements from a project would also be nice.

    Best regards,

    ReplyDelete
  10. Dear Boris,

    I'm in charge of validating RTC through a Proof of Concept, but I'm unfortunately starting to discover the RTC subject.
    We want to validate that RTC could cover the sames Lifecycle areas than Microsoft Team Foundation Server.

    We took a long time to setup correctly the server, then to reach RTC from Visual Studio 2010!

    Now, I face problems in reaching the source code loaded in RTC from a basic user account. Note: it's ok using my admin account.


    I think something is not correctly setup within the Step 2 of your above tutorial session (part I).
    Could you help me in finding an userfriendly tutorial about configuring a Project Area (and Team/Stream Areas) from the RTC Web Admin interface ?

    Note: The Eclipse Project Area wizard discribed in the tutorial "http://pic.dhe.ibm.com/infocenter/rtc/v1r0m0/index.jsp?topic=%2Fcom.ibm.team.concert.tutorial.doc%2Ftopics%2Ftut_rtc_scm.html" misses in Visual Studio.

    Thanks in advance.
    Regards.
    O.L.

    ReplyDelete
    Replies
    1. Yes, you will need to create the project from the web client which can be reached at:
      https://:/ccm/admin

      The host and port are the same as the ones you used to establish our repository connection in VS. If you cannot log into this interface with your other user, ensure that your user has the appropriate licence CAL assiged and that they, at least, have the JazzProjectAdmin type of role. You can find these settings here:
      https://:/jts/admin in the Users section. The information center your are using is quite old. Try this one:

      http://publib.boulder.ibm.com/infocenter/clmhelp/v3r0m1/index.jsp

      In particular, perhaps start at the "Getting started with project areas and lifecycle projects" section.

      Delete
  11. OK. Thank you.
    I'll read first the newest tutorial in reference.
    Regards
    O.L.

    ReplyDelete
  12. Hi Boris,

    I created a sample C# program based on your code, to do the steps:
    - get requirement services from RM
    - get requirement catalog, then project services
    - get requirement factory
    - create a xml of sample requirment, then send a PUT request to the factory URL, then I can create the new requirment well.
    - however, when I get the requirement back, modify the connent such as adding a new properties, then PUT it back to the requirement URL, I always receive the error 403: forbidden...


    if you could help me how to fix the issue, it would be great!
    I have installed JTS-CCM-RM-repo-4.1Beta1 version on Window 7.

    Thanks,
    Huy.

    ReplyDelete
    Replies
    1. It could be a bug.. try posting this question on the jazz.net/forums page. I also am monitoring it and perhaps somebody has had the same issue. You can try the "Extending Team Concert" forum or the RRC one.

      Delete