Slow Gitlab Integration connectionSlow Gitlab Integration connectionhttps://www.klaros-testmanagement.com/de/c/message_boards/find_thread?p_l_id=11703&threadId=4837232024-03-29T11:41:57Z2024-03-29T11:41:57ZRE: Slow Gitlab Integration connectionTorsten Stolpmannhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4850232020-03-09T21:38:59Z2020-03-09T21:38:59ZHello Christopher,<br /><br />our automated tests are running a login, issue creation and retrieval in GitLab in roughly 2.5 sec (with GitLab running in a virtualized environment on crap hardware). I am unsure on what we can do on our side to remedy your situation.<br /><br />Can you rule out SSL-negotiation problems with your GitLab installation? If it is exposed to the internet <a href="https://www.ssllabs.com/ssltest/analyze.html">https://www.ssllabs.com/ssltest/analyze.html</a> might help to diagnose.<br /><br />We will keep an eye on this, but I am missing the right idea here.<br /><br />Torsten<br /><br />P.S.: I am travelling this week, so do not expect fast responses.Torsten Stolpmann2020-03-09T21:38:59ZRE: Slow Gitlab Integration connectionChristopher Huberhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4850012020-03-09T08:02:11Z2020-03-09T08:02:11ZDid you found anything related this topic?<br />or was you able to reproduce the issue?Christopher Huber2020-03-09T08:02:11ZRE: Slow Gitlab Integration connectionTorsten Stolpmannhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4839092020-02-28T16:05:34Z2020-02-28T16:05:34ZThanks for the detailed report.<br /><br />The <span style="font-size: 12px;"> </span><strong><span style="font-size: 12px;">/api/v4/projects/182/issues/</span></strong><span style="color: #ff0000"><span style="font-size: 12px;">2147483647</span></span> request is a probe for validating the connection parameters.<br />It should only be done once per user session to a Gitlab project and is intentional.<br /><br />I am unsure about the error message regarding the authentication error you receive, I am going to validate this and get back to you.Torsten Stolpmann2020-02-28T16:05:34ZRE: Slow Gitlab Integration connectionChristopher Huberhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4838912020-02-28T07:37:45Z2020-02-28T07:37:45ZHello again,<br /><br />you are right. 3 seconds is to long to wait. but it is better than 30 <img alt="emoticon" src="https://www.klaros-testmanagement.com/o/khorne-theme/images/emoticons/smile.gif" ><br /><br /><br />Thank you for that modified version... but it is still not working. Now I got another issue:<br /><br /><strong>The Authentication is wrong.<br /></strong><br />I looked into the logs and there is a strange behavior to check if the gitlab server is available.<br /><br />1. A request is send to: <strong>/api/v4/projects/182/issues/</strong><span style="color: #ff0000">2147483647</span> with an API Token (which is valid).<br />This is ok, but I'm confused about the issue-id <strong>2147483647</strong>. This issue doesn't exists so the server get an <span style="color: #0000ff">404 not found</span>.<br /><br />2. Then a second request is send to: <strong>/api/v4/user</strong> which looks good, BUT there is <strong>NO</strong> <strong>ApiToken</strong> in the request. So the Server got an <span style="color: #0000ff">401 Unauthorized<br /><br /></span>More requests are not possible because of an exception, that the<strong> username/token is invalid</strong>, which is not the case.Christopher Huber2020-02-28T07:37:45ZRE: Slow Gitlab Integration connectionTorsten Stolpmannhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4838772020-02-27T13:40:45Z2020-02-27T13:40:45ZThank you for the analysis, this is interesting.<br /><br />First of all, I honestly think that 3 seconds for a single request is unacceptable in any case.<br /><br />I am unsure if this is based in our changes regarding connection pooling applied, so I added a modified version of the connector to this post. Please replace this file (located in webapps/klaros-web/WEB-INF/lib) restart the server and retry.<br /><br />If this still does not significantly better your situation then I strongly suspect a cause outside of Klaros.<br /><br />Sadly, this will reintroduce issue KLAROS-4298 (Unable to select an assignee when creating/editing GitLab issues) so this is not a permanent solution.<br /><br />Please let me know your results.Torsten Stolpmann2020-02-27T13:40:45ZRE: Slow Gitlab Integration connectionChristopher Huberhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4838492020-02-27T09:16:56Z2020-02-27T08:53:43ZI've now logged the following action:<br /><br />Open a <strong>Testcase,</strong> hit <strong>"Issue" </strong>and select<strong> "new"</strong>. (execution time untill the UI opens: ca. 30 seconds)<strong><br /></strong>Then I got the following output: (see attachment)<br /><br />You can see that there are 5 Messages which are repeated 5 times.<br /><br />One round tooks 3 seconds. So if this is repeated 5 times with some delay between the requests, then we got this 20 till 30 seconds.<br /><br />But what can we do now?? I'm totally fine with the 3 seconds for one round... but it is not so nice that this occurs 5 times.Christopher Huber2020-02-27T08:53:43ZRE: Slow Gitlab Integration connectionTorsten Stolpmannhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4838232020-02-26T14:32:00Z2020-02-26T14:08:56ZLet me have a look ...<br /><br />The file to edit is: <span style="font-family: courier new, courier, monospace">webapps/klaros-web/WEB-INF/classes/logback.xml</span><br /><br />We are using logback with slf4j. jul-to-slf4j is already present. The GitLab API uses the standard Java HttpURLConnection class.<br /><br />You will probably have to change the default log level and add the correct log level settings.<br /><br />I was not yet able to try this out.Torsten Stolpmann2020-02-26T14:08:56ZRE: Slow Gitlab Integration connectionChristopher Huberhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4838162020-02-26T14:11:30Z2020-02-26T14:03:33ZI've installed Klaros 4.12.5 on my local machine and captured the network traffic with Wireshark. Sadly I'm using https, so I can't see the complete request but I found some interesing packages:<br /><br />First I got a request to gitlab with a Hello packets and some data, than after some seconds I got an reset signal:<br /><div class="lfr-code"><table><tbody><tr><td class="line-numbers" data-line-number="1"></td><td class="lines"><div class="line">No. Time Protocol Length Info</div></td></tr><tr><td class="line-numbers" data-line-number="2"></td><td class="lines"><div class="line"><br /></div></td></tr><tr><td class="line-numbers" data-line-number="3"></td><td class="lines"><div class="line">89 4.652329 TLSv1.2 647 Client Hello</div></td></tr><tr><td class="line-numbers" data-line-number="4"></td><td class="lines"><div class="line">93 5.174111 TLSv1.3 140 Application Data</div></td></tr><tr><td class="line-numbers" data-line-number="5"></td><td class="lines"><div class="line">94 5.174495 TLSv1.3 379 Application Data</div></td></tr><tr><td class="line-numbers" data-line-number="6"></td><td class="lines"><div class="line"><br /></div></td></tr><tr><td class="line-numbers" data-line-number="7"></td><td class="lines"><div class="line">139 8.234023 TCP 54 60523 → 443 [RST] Seq=1049 Win=0 Len=0</div></td></tr></tbody></table></div><br /><br />Then the request repeats with the same Data and length. So there are again packets with data (same as above). <br /><br /><div class="lfr-code"><table><tbody><tr><td class="line-numbers" data-line-number="1"></td><td class="lines"><div class="line">No. Time Protocol Length Info</div></td></tr><tr><td class="line-numbers" data-line-number="2"></td><td class="lines"><div class="line"><br /></div></td></tr><tr><td class="line-numbers" data-line-number="3"></td><td class="lines"><div class="line">270 11.264539 TLSv1.3 647 Client Hello TCP 54</div></td></tr><tr><td class="line-numbers" data-line-number="4"></td><td class="lines"><div class="line">274 11.301455 TLSv1.3 140 Application Data</div></td></tr><tr><td class="line-numbers" data-line-number="5"></td><td class="lines"><div class="line">275 11.301778 TLSv1.3 379 Application Data</div></td></tr><tr><td class="line-numbers" data-line-number="6"></td><td class="lines"><div class="line"><br /></div></td></tr><tr><td class="line-numbers" data-line-number="7"></td><td class="lines"><div class="line">339 14.119048 TCP 54 60556 → 443 [RST] Seq=1049 Win=0 Len=0</div></td></tr></tbody></table></div><br />This looks a littlebit strange for me.... Is there a way to log the messages from Klaros? Maybe with another log level in any property file?Christopher Huber2020-02-26T14:03:33ZRE: Slow Gitlab Integration connectionChristopher Huberhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4838052020-02-26T12:24:27Z2020-02-26T12:24:27ZI'm using the latest Klaros Version 4.12.5. running in a docker container. And also gitlab is running in a docker container.<br /><br />Currently I've no option for wireshark or something like that. Because of the dockerized structure. <br /><br />But I will try to get the HTTP communication, so that we can see what happens.Christopher Huber2020-02-26T12:24:27ZRE: Slow Gitlab Integration connectionTorsten Stolpmannhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4837942020-02-26T12:17:00Z2020-02-26T12:17:00ZTo answer your question: Yes, there are multiple requests going on when creating an issue on to the GitHub server, mostly for retrieving project metadata (Milestones, Priorities, etc.). Some of them may be repeated.<br /><br />You did not state the version of Klaros you are using, this is of importance here.<br /><br />We are leveraging a third party software to connect to Gitlab. Before version 4.12.5, this implementation connected to all projects on the GitLab server, we patched that behavior in 4.12.5, so only the configured project will be retrieved. I am suspecting that this may be the case here.<br /><br />So if you are using a version below 4.12.5 please update and retry.<br /><br />On the downside we also had to remove connection caching in 4.12.5 as it was interfering with the ability to select an assignee during issue creation. This may as well have a detrimental influence here, but I wouldn't expect this in this magnitude.<br /><br />If you are experienced with using Wireshark or a logging http/https proxy I would be interested to see what is going on at comunication with your Gitlab server.<br /><br />Regards,<br /><br />Torsten<br /> Torsten Stolpmann2020-02-26T12:17:00ZRE: Slow Gitlab Integration connectionChristopher Huberhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4837872020-02-26T11:58:35Z2020-02-26T11:58:35ZCould it happen, that the request will be send multiple times?? I mean, after I've send a new Issue to gitlab, then Klaros retrieves the new information about the inserted issue. But could it happen, that this information request will be triggered multiple times? Christopher Huber2020-02-26T11:58:35ZRE: Slow Gitlab Integration connectionChristopher Huberhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4837802020-02-26T11:45:47Z2020-02-26T11:45:47ZIf I try to call the Gitlab API with a CURL command, than I can create an issue in under 2seconds. Also when I try to GET some informtion about the group or project. (i.e. the members or the milestones). This is also done in a few seconds.<br /><br />So please, can you tell me, how you retrieve the information from gitlab and how you create a new issue?<br /><br />Thank you very much.Christopher Huber2020-02-26T11:45:47ZRE: Slow Gitlab Integration connectionChristopher Huberhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4837692020-02-26T10:53:59Z2020-02-26T10:53:59ZHello Torsten,<br /><br />I've 260 Projects in 3 different groups.<br />The AccessToken I've added in Klaros is from a user which has only access to ONE project. This is also the project I've added in the configuration of Klaros.<br /><br />Regards,<br />ChristopherChristopher Huber2020-02-26T10:53:59ZRE: Slow Gitlab Integration connectionTorsten Stolpmannhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4837492020-02-26T10:51:30Z2020-02-26T10:51:30ZHello Christopher,<br /><br />can you tell me how many projects are hosted on this server (roughly)?<br /><br />Regards,<br /><br />TorstenTorsten Stolpmann2020-02-26T10:51:30ZSlow Gitlab Integration connectionChristopher Huberhttps://www.klaros-testmanagement.com/de/c/message_boards/find_message?p_l_id=11703&messageId=4837222020-02-26T10:38:16Z2020-02-26T10:38:04ZHello everybody,<br /><br />I'm working with the community edition of the Klaros-Testmanagement and I want to integrate it with Gitlab to create Issues.<br /><br />I've added a Gitlab Integration in the config menu and then I linked it to a project. Until now, everything is fine.<br /><br />BUT:<br />If I now want to create a <strong>new Issue</strong> (it doesn't matter if I create it after a test or directly in the testcase), then this takes up to <span style="color: #ff0000"><strong>40 seconds</strong></span> to finish the request. The first slow down starts if I click on "New Issue". Then the <strong>UI doesn't response for 10 seconds</strong> and then I see the UI to add a new Issue. After filling out everything I <strong>hit "Save" and it takes 30 seconds</strong> to response "Issue successfully created".<br /><br />If I take a look on the gitlab server, then I see, that there is a high CPU workload. (ca. 50%). So now I'm wondering, whats happen there.<br />The problem is, that I can't see how many request are send from Klaros to Gitlab or back. So can you help me to understand the communication between Klaros and Gitlab, to find out if the problem is on the Klaros side or on the Gitlab side.<br /><br />Kind regards<br />ChristopherChristopher Huber2020-02-26T10:38:04Z