{"id":5788,"date":"2021-01-12T08:00:43","date_gmt":"2021-01-12T08:00:43","guid":{"rendered":"https:\/\/sandbox.weareadaptive.com\/?p=5788"},"modified":"2021-06-17T11:18:57","modified_gmt":"2021-06-17T10:18:57","slug":"acceptance-testing-with-accelerators","status":"publish","type":"post","link":"https:\/\/sandbox.weareadaptive.com\/fr\/2021\/01\/12\/acceptance-testing-with-accelerators\/","title":{"rendered":"Acceptance Testing with Adaptive&rsquo;s Accelerators"},"content":{"rendered":"<link rel=\"preconnect\" href=\"https:\/\/fonts.gstatic.com\">\n<link href=\"https:\/\/fonts.googleapis.com\/css2?family=Source+Sans+Pro:wght@300;600&#038;display=swap\" rel=\"stylesheet\">\n<style>\t\/* Source Sans Pro Light *\/\n\t\/*body{\n\t  font-family: \"Source Sans Pro Light\";\n\t  font-size: 16px;\n\t  font-weight: 300;\n\t}\n\th1, h2, h3, h4, h5, h6{\n\t  font-family: \"Source Sans Pro SemiBold\";\n\t  font-weight: 600;\n\t}*\/\/* Inter *\/body, p{\n\t  font-family: 'Source Sans Pro', sans-serif !important;\n\t  font-size: 16px;\n\t  font-weight: 300;\n\t}h1, h2, h3, h4, h5, h6{\n\t  font-family: 'Source Sans Pro', sans-serif !important;\n\t  font-weight: 600 !important;\n\t}<\/style>\n<div style=\"text-align: justify;\">At Adaptive we believe that QA and Testing are at the core of any agile delivery and your testing approach cannot be truly agile without test automation. So when we invested in our accelerator technology, we realised that an acceptance testing solution must be a part of it and it must be done right.<\/div>\n<p>What we needed was a test framework which:<\/p>\n<ul>\n<li style=\"text-align: justify;\">Promotes collaboration within the team<\/li>\n<li style=\"text-align: justify;\">Makes it easy to concentrate on testing business features the same way our accelerators allow us to concentrate on writing business code<\/li>\n<li style=\"text-align: justify;\">Helps us write fast and reliable tests<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">In this blog post, I would like to share some of the most exciting features of the framework that allow us to achieve these goals.<\/p>\n<p>But first, some technical information.<\/p>\n<p>Our framework is written in Java and is based on Cucumber to make collaboration between QA, DEV and BA easier, and to decrease the risk of getting complex business functionality wrong.<\/p>\n<p>Now let\u2019s talk about the test framework features.<\/p>\n<h2>Test Execution Mode<\/h2>\n<div style=\"text-align: justify;\">\n<p>Test Execution Mode is one of the most powerful features for me as a QA! You can use the framework for any kind of test: component or integration, but you get the best value when using it for end-to-end (e2e) tests, which are notoriously most useful from the business perspective and at the same time least performant and least stable from the execution perspective.<\/p>\n<p>But what if we could make e2e tests really fast and stable? Now we can!<\/p>\n<\/div>\n<p>The framework allows you to run tests in 3 modes:<\/p>\n<ul>\n<li><span style=\"text-align: justify;\"><span style=\"color: #27578c;\"><strong>Existing<\/strong><\/span> &#8211; this is the usual way that e2e tests are executed in the industry &#8211; against a previously deployed environment.<\/span><\/li>\n<li><span style=\"text-align: justify;\"><span style=\"color: #27578c;\"><strong>Embedded real<\/strong><\/span> &#8211; in this mode the tests and the services under test run on separate threads within the same JVM, communicating using real protocols, like HTTP, FIX, and Aeron.<\/span><\/li>\n<li><span style=\"text-align: justify;\"><span style=\"color: #27578c;\"><strong>Embedded direct<\/strong><\/span> &#8211; this is the e2e supermode, where the tests and the services not only run in the same JVM, but are wired together directly, bypassing the underlying network protocols, and running in a single thread. This makes the tests completely consistent, and lightning fast, allowing us to execute 107 scenarios with 1182 steps tests in just 44 seconds on one of our projects.<\/span><\/li>\n<\/ul>\n<div style=\"text-align: justify;\">\n<p>Now the usual test pyramid can be extended and e2e layer refined:<\/p>\n<p>We only need to run a subset of e2e tests against a deployed environment &#8211; those that could actually be affected by production-like wiring and topology.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-5791 size-full\" src=\"https:\/\/sandbox.weareadaptive.com\/wp-content\/uploads\/2020\/12\/Adaptive_E2E-Test-Pyramid.png\" alt=\"\" width=\"464\" height=\"364\" srcset=\"https:\/\/sandbox.weareadaptive.com\/wp-content\/uploads\/2020\/12\/Adaptive_E2E-Test-Pyramid.png 464w, https:\/\/sandbox.weareadaptive.com\/wp-content\/uploads\/2020\/12\/Adaptive_E2E-Test-Pyramid-300x235.png 300w\" sizes=\"(max-width: 464px) 100vw, 464px\" \/><\/p>\n<p>One of the reasons that e2e tests have a bad reputation is their reliability. It is often not just due to asynchronous behavior of the system which the test execution mode deals with, but also the stability of the environment: does it have all the pieces that I need? Does it have all the test data that I need?<\/p>\n<p>In the embedded real and direct modes, the test framework spins up a new \u201cenvironment\u201d for you containing just the pieces necessary for the tests. You can inject the test data at the start of each test case saving us from the <a href=\"http:\/\/xunitpatterns.com\/Obscure%20Test.html#Mystery%20Guest\" target=\"_blank\" rel=\"noopener noreferrer\">\u201cmystery guest\u201d antipattern<\/a>.<\/p>\n<\/div>\n<p>Here is how it looks in a feature file:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"asm\" data-enlighter-theme=\"droide\" data-enlighter-linenumbers=\"false\">Scenario: CreateAsset\r\nGiven the services 'engine, admin-gateway'\r\nAnd admin session Foo successfully logs in with username 'lidia'\r\nWhen admin session Foo successfully creates asset 'USD' 'US Dollar'\r\nAnd admin session Foo successfully creates asset 'GBP' 'British Pound'\r\nThen admin session Foo can see assets:\r\n| Code | Name |\r\n| USD | US Dollar |\r\n| GBP | British Pound |<\/pre>\n<h2>Test clients and utils out of the box<\/h2>\n<div style=\"text-align: justify;\">\n<p>One of the key features of our technical accelerators is code generation, and we leverage it for the test automation solution as well. When you start a project, you get generated test clients for each gateway (like FIX or Web) out of the box covering all the API calls.<\/p>\n<p>For example, if we look at the step from the scenario above:<\/p>\n<\/div>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"asm\" data-enlighter-theme=\"droide\">Then\u00a0admin\u00a0session\u00a0Foo\u00a0can\u00a0see\u00a0assets:\r\n| Code | Name |\r\n| USD | US Dollar |\r\n| GBP | British Pound |<\/pre>\n<p>The implementation that DEV or QA created looks like:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"asm\" data-enlighter-theme=\"droide\" data-enlighter-linenumbers=\"false\">@Then(\"admin session {adminSession} can see assets:\")\r\npublic void awaitAsset(final AdminGatewaySessionDriver session,\r\nfinal DataTable expectedAssets)\r\n{\r\nfinal AssetServiceProxy assetServiceProxy = session.services().getAssetServiceProxy();\r\nfinal AssetServiceClientRecorder assetRecorder = session.services().getAssetServiceClientRecorder();\r\n\r\nfinal long correlationId = assetServiceProxy.generateCorrelationId();\r\nassetServiceProxy.getAllAssets(correlationId);\r\n\r\ndeployment.expect(assetRecorder.getAllAssetsResponse())\r\n.withCorrelationId(correlationId)\r\n.toHaveCompleted()\r\n.toContainExactly(expectedAssets);\r\n}<\/pre>\n<div style=\"text-align: justify;\">\n<p>Every line in this step implementation code includes either a call to generated code; for example:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"asm\" data-enlighter-theme=\"droide\" data-enlighter-linenumbers=\"false\">session.services().getAssetServiceClientRecorder();<\/pre>\n<div style=\"text-align: justify;\">\n<p>or a platform util; for example:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"asm\" data-enlighter-theme=\"droide\" data-enlighter-linenumbers=\"false\">deployment.expect()<\/pre>\n<div style=\"text-align: justify;\">\n<p>The deployment.expect() accepts both objects and datatables for verification and, in the latter case, the datatable transformation and comparison is happening under the hood.<\/p>\n<\/div>\n<\/div>\n<h2>FIX protocol support<\/h2>\n<p>A full-featured FIX gateway is part of our accelerator set, so it\u2019s only natural that its test automation framework allows you to send and verify FIX messages.<\/p>\n<p>Over the years I have worked with numerous solutions for FIX testing, the protocol versions and messages vary a lot between our clients so we had to find a generic solution that would fit if not all, then most cases. The easiest approach was to expose FIX message at the gherkin layer, here is an example test:<\/p>\n<\/div>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"asm\" data-enlighter-theme=\"droide\" data-enlighter-linenumbers=\"false\">Scenario: FIX session subscribes to market data on connect and sends a snapshot request\r\nGiven the services 'engine, market-data-gateway, ref-data-injector'\r\nWhen the market data provider session 'SPOT-FX' connects\r\nThen the market data provider session 'SPOT-FX' receives a message containing:\r\n\"\"\"\r\n[MsgType] 35 = V [MARKET_DATA_REQUEST]\r\n[MDReqID] 262 = CAPTURE_AS: MD_REQ_ID_1\r\n[SubscriptionRequestType] 263 = 0 [SNAPSHOT]\r\n[MarketDepth] 264 = 1\r\n[NoMDEntryTypes] 267 = 2\r\n[MDEntryType] 269 = 0 [BID]\r\n[MDEntryType] 269 = 1 [OFFER]\r\n[NoRelatedSym] 146 = 1\r\n[Symbol] 55 = GBP\/USD\r\n\"\"\"<\/pre>\n<div style=\"text-align: justify;\">\n<p>This approach makes it exceptionally easy for the team to start using the test framework for any specific FIX setup on a project and then later on they can adjust it if needed.<\/p>\n<h2>Time Travel<\/h2>\n<p>The last but not least feature that I would like to share in this article is ability to time travel (again, another case where e2e tests can be really cumbersome). This feature is, of course, only supported in embedded modes.<\/p>\n<p>Here is how it looks at the gherkin layer:<\/p>\n<\/div>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"asm\" data-enlighter-theme=\"droide\" data-enlighter-linenumbers=\"false\">Scenario: Scheduled Job\r\nGiven the services 'engine, admin-gateway'\r\nGiven admin session Foo successfully logs in with username 'lidia'\r\nWhen admin session Foo successfully creates a reminder in 60 seconds saying 'Hi there'\r\nAnd time advances 1 minute\r\nThen admin session Foo sees these reminders:\r\n| Type | Body |\r\n| Reminder | Hi there |\r\n\r\n<\/pre>\n<p>And here is the implementation for the \u201ctime advances\u201d step:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"asm\" data-enlighter-theme=\"droide\" data-enlighter-linenumbers=\"false\">@Given(\"time advances {int} {timeUnit}\")\r\npublic void timeAdvances(final int seconds, final TimeUnit timeUnit)\r\n{\r\n   timeMachine.advanceBy(timeUnit.toMillis(seconds));\r\n}\r\n<\/pre>\n<div style=\"text-align: justify;\">\n<p>The time machine is another util supplied by the framework that allows to manipulate the cluster clock and avoid waiting in this kind of tests.<\/p>\n<h2>Summary<\/h2>\n<p>When our clients choose an accelerated solution delivery from Adaptive, they not only get tools that support development but also tools that support QA (and UX and DevOPS). This allows us to not just write code faster but to do so with the confidence that business requirements are met and any risk of regression is minimized.<\/p>\n<h2>What\u2019s next<\/h2>\n<p>Our test automation solution is already quite mature and helping us to significantly speed up delivery for several of our clients, but there are always more things to add:<br \/>\nWeb UI test automation.<\/p>\n<p>Further improvement of platform utils to streamline expectations and assertions.<\/p>\n<p>If you would like to know more about it, please don\u2019t hesitate to reach out to me on <a href=\"https:\/\/www.linkedin.com\/in\/lidia-sinitsyna-a9654140\/\" target=\"_blank\" rel=\"noopener noreferrer\">LinkedIn<\/a> or click the button below.<\/p>\n<\/div>\n<p>&nbsp;<\/p>\n<p style=\"text-align: right;\"><img loading=\"lazy\" decoding=\"async\" class=\"size-thumbnail wp-image-4061 alignright\" src=\"https:\/\/sandbox.weareadaptive.com\/wp-content\/uploads\/2020\/12\/Lidia-Sinitsyna_round.png\" alt=\"\" width=\"150\" height=\"150\" \/><\/p>\n<h1 style=\"text-align: right;\">Lidia Sinitsyna<\/h1>\n<p style=\"text-align: right;\">Head of QA,<br \/>\nAdaptive Financial Consulting Ltd<\/p>\n<p style=\"text-align: right;\"><button class=\"waves-effect waves-light btn-flat alt cta-talk\">Let\u2019s talk<\/button><\/p>\n","protected":false},"excerpt":{"rendered":"<p>At Adaptive we believe that QA and Testing are at the core of any agile delivery and your testing approach &#8230;<\/p>\n","protected":false},"author":24,"featured_media":5815,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6,138,147],"tags":[233,250],"class_list":["post-5788","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog","category-accelerators","category-qa-testing","tag-accelerators","tag-acceptance-testing"],"_links":{"self":[{"href":"https:\/\/sandbox.weareadaptive.com\/fr\/wp-json\/wp\/v2\/posts\/5788"}],"collection":[{"href":"https:\/\/sandbox.weareadaptive.com\/fr\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sandbox.weareadaptive.com\/fr\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sandbox.weareadaptive.com\/fr\/wp-json\/wp\/v2\/users\/24"}],"replies":[{"embeddable":true,"href":"https:\/\/sandbox.weareadaptive.com\/fr\/wp-json\/wp\/v2\/comments?post=5788"}],"version-history":[{"count":26,"href":"https:\/\/sandbox.weareadaptive.com\/fr\/wp-json\/wp\/v2\/posts\/5788\/revisions"}],"predecessor-version":[{"id":6758,"href":"https:\/\/sandbox.weareadaptive.com\/fr\/wp-json\/wp\/v2\/posts\/5788\/revisions\/6758"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sandbox.weareadaptive.com\/fr\/wp-json\/wp\/v2\/media\/5815"}],"wp:attachment":[{"href":"https:\/\/sandbox.weareadaptive.com\/fr\/wp-json\/wp\/v2\/media?parent=5788"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sandbox.weareadaptive.com\/fr\/wp-json\/wp\/v2\/categories?post=5788"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sandbox.weareadaptive.com\/fr\/wp-json\/wp\/v2\/tags?post=5788"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}