{"id":51610,"date":"2017-08-25T11:54:19","date_gmt":"2017-08-25T06:24:19","guid":{"rendered":"http:\/\/www.tothenew.com\/blog\/?p=51610"},"modified":"2017-10-12T16:00:51","modified_gmt":"2017-10-12T10:30:51","slug":"how-to-build-smart-tv-apps-with-testing-strategy","status":"publish","type":"post","link":"https:\/\/www.tothenew.com\/blog\/how-to-build-smart-tv-apps-with-testing-strategy\/","title":{"rendered":"How to Build Winning Smart TV Apps With a Comprehensive Testing Strategy?"},"content":{"rendered":"<p><span style=\"font-weight: 400\">The pattern of content consumption has changed markedly in the last two decade. <\/span><span style=\"font-weight: 400\">From mobile phones in the 90\u2019s to smart phones to smart TVs, the demand for smart connected devices continues to grow. <\/span><span style=\"font-weight: 400\">According to a <\/span><a title=\"Gartner Report\" href=\"http:\/\/www.gartner.com\/newsroom\/id\/2934717\" target=\"_blank\"><span style=\"font-weight: 400\">Gartner<\/span><\/a><span style=\"font-weight: 400\"><a title=\"Gartner Report\" href=\"http:\/\/www.gartner.com\/newsroom\/id\/2934717\" target=\"_blank\"> consumer survey<\/a>, worldwide, by 2018, 87 percent of the TVs shipped annually will be smart TVs. <\/span><span style=\"font-weight: 400\">The figure below indicates the steep rise in the demand for smart TV from 2011 to 2017.<\/span><\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-51612\" src=\"\/blog\/wp-ttn-blog\/uploads\/2017\/08\/pic1.png\" alt=\"pic\" width=\"1000\" height=\"716\" srcset=\"\/blog\/wp-ttn-blog\/uploads\/2017\/08\/pic1.png 1000w, \/blog\/wp-ttn-blog\/uploads\/2017\/08\/pic1-300x214.png 300w, \/blog\/wp-ttn-blog\/uploads\/2017\/08\/pic1-624x446.png 624w\" sizes=\"(max-width: 1000px) 100vw, 1000px\" \/><\/p>\n<p>Source: Frost &amp; Sullivan<\/p>\n<h2><b>Brands are foraying into Smart TV applications\u00a0<\/b><\/h2>\n<p><span style=\"font-weight: 400\">More and more brands are foraying into the smart TV app market. Entertainment and games are the biggest chunks of apps on Smart TV app stores, followed by social network and shopping apps. <\/span><a title=\"Reasons Why Apple TV is taking the World by Storm\" href=\"http:\/\/www.tothenew.com\/blog\/5-reasons-why-apple-tv-is-taking-the-world-by-storm\/\"><span style=\"font-weight: 400\">Apple TV<\/span><\/a><span style=\"font-weight: 400\"><a title=\"Reasons Why Apple TV is taking the World by Storm\" href=\"http:\/\/www.tothenew.com\/blog\/5-reasons-why-apple-tv-is-taking-the-world-by-storm\/\"> is scaling up big time<\/a>, and Android TV has seen about 5-10 million app installs to this day.<\/span><\/p>\n<p><span style=\"font-weight: 400\">Great <\/span><a title=\"User experience design services\" href=\"http:\/\/www.tothenew.com\/experience-design\"><span style=\"font-weight: 400\">user experience<\/span><\/a><span style=\"font-weight: 400\"> is the definitive factor for the success of TV apps. Faulty apps with errors and bugs can play havoc with a brand\u2019s reputation. Therefore, it is important that app developers conduct rigorous testing before pushing their apps on smart TV app stores, as the cost associated with testing is way less than the cost accrued in finding and fixing bugs and errors post production.<\/span><\/p>\n<h2><b>Achieving Quality in<\/b> <b>Smart TV Application Development<\/b><\/h2>\n<p><span style=\"font-weight: 400\">Testing Smart TV<\/span><span style=\"font-weight: 400\"> apps is a different ball game altogether when compared to testing mobile and tablet apps. Therefore a significant shift in testing approach needs to be made for testing and QA of smart TV apps.<\/span><\/p>\n<p>User perspective is an important <a title=\"Top 6 Key Considerations When Developing Smart TV Application\" href=\"http:\/\/www.tothenew.com\/blog\/top-6-key-considerations-when-developing-smart-tv-application\/\">consideration for developing smart TV apps<\/a>. Hence interface testing with people is important. While automated testing and emulators have become a norm nowadays given the amount of time it saves for testing the varied use cases, manual testing should be complemented to consider factors that automated test environments cannot. The size of the screen, distance between the user and the screen are important considerations while testing the visual rendering, image size, and placement of elements on the screen. Emulators are not as effective when testing design and layout. Therefore it is always better to use real TV. It is important to test out the UI in multiple settings.<\/p>\n<p><span style=\"font-weight: 400\">What makes<\/span> <a title=\"Smart TV app testing\" href=\"http:\/\/www.tothenew.com\/testing\/automated-independent-manual-testing\"><span style=\"font-weight: 400\">Smart TV apps testing<\/span><\/a><span style=\"font-weight: 400\"> more complex is the availability of a sheer number of smart TV vendors, numerous versions, and models, the number of testing platforms and Operating Systems. It is essential for app providers to get their application tested for the particular Smart TV model, the platform used and the operating system\/ browser that it supports.<\/span><\/p>\n<h2><b>Typical Test Types<\/b><\/h2>\n<ul>\n<li style=\"font-weight: 400\"><b>Regression testing<\/b><span style=\"font-weight: 400\">: It is important for detecting errors or bugs in functional or non-functional areas that might arise after code modification<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Functional testing: <\/b><span style=\"font-weight: 400\">\u00a0It is a <a title=\"A Must to Kick Off QA Process\" href=\"http:\/\/www.tothenew.com\/blog\/domain-knowledge-a-must-to-kick-off-qa-process\/\">QA process<\/a> and a type of black-box testing. It tests the specifications of software components and typically focuses if the output is as per requirement<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Smoke testing:<\/b><span style=\"font-weight: 400\"> Although such a testing is not very in-depth, it is preferable to pass all QA builds through smoke testing to weed out basic errors<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Browser compatibility testing:<\/b><span style=\"font-weight: 400\"> App is thoroughly checked under different versions of browsers<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Performance testing of API\u2019s: <\/b><span style=\"font-weight: 400\">API<\/span> <span style=\"font-weight: 400\">integration testing and performance testing can be done <a title=\"Things You Must Know About Load-Testing Using JMeter\" href=\"http:\/\/www.tothenew.com\/blog\/7-things-you-must-know-about-load-testing-using-jmeter\/\">using JMeter<\/a><\/span><\/li>\n<li style=\"font-weight: 400\"><b>Usability Testing: <\/b><span style=\"font-weight: 400\">Usability can make or break the user experience. With<\/span> <span style=\"font-weight: 400\">usability testing<\/span><a href=\"http:\/\/www.tothenew.com\/blog\/usability-testing-for-smart-tv-quick-tips-and-considerations\/\"><span style=\"font-weight: 400\">.<\/span><\/a><span style=\"font-weight: 400\"> the application\u2019s flow and its navigation is checked, and UX is improved<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Crowd Testing: <\/b><span style=\"font-weight: 400\">Multiple users test the app for wider inputs on errors<\/span><\/li>\n<\/ul>\n<h2><b>Considerations for <\/b><b>Smart TV Application Testing<\/b><\/h2>\n<ul>\n<li style=\"font-weight: 400\"><i><span style=\"font-weight: 400\">Abide by Platform Store Guidelines<\/span><\/i><span style=\"font-weight: 400\">&#8211; All Platform Stores have guidelines that applications need to follow before they can be listed on the App Store. It is of primary importance for testers and developers to abide by these guidelines to avoid application rejection. Most companies have basic guidelines such as <\/span><span style=\"font-weight: 400\">having instant and distinct control feedback, <\/span><span style=\"font-weight: 400\">not keeping unnecessary screen levels, keeping consistency in navigation, screen layouts, controls and many others. <\/span><span style=\"font-weight: 400\">Here are guidelines for<\/span><a title=\"Smart TV app\" href=\"https:\/\/sdkdocs.roku.com\/display\/sdkdoc\/Pre-certification+Check+List\"><span style=\"font-weight: 400\"> Roku<\/span><\/a><span style=\"font-weight: 400\">,<\/span><a title=\"Smart TV app\" href=\"https:\/\/developer.apple.com\/app-store\/review\/guidelines\/\"> <span style=\"font-weight: 400\">Apple TV<\/span><\/a><span style=\"font-weight: 400\">,<\/span><a title=\"Smart TV app\" href=\"https:\/\/developer.tizen.org\/community\/tip-tech\/validation-guidelines\"> <span style=\"font-weight: 400\">Tizen<\/span><\/a>,<span style=\"font-weight: 400\"> and<\/span><a title=\"Smart TV app\" href=\"https:\/\/developer.android.com\/distribute\/best-practices\/launch\/distribute-tv.html\"> <span style=\"font-weight: 400\">Android TV<\/span><\/a>&nbsp;<\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\"><span style=\"font-weight: 400\"><em>Maintain bugs in a tracking tool<\/em> &#8211; It is important to keep track of all issues and maintain them at a centralized database or use a <a title=\"Effective techniques for Bug Reporting and Bug Tracking\" href=\"http:\/\/www.tothenew.com\/blog\/effective-techniques-for-bug-reporting-and-bug-tracking\/\">bug tracking and management<\/a> tool. It will help to ensure that the bugs are fixed timely<\/span><\/span>&nbsp;<\/li>\n<li style=\"font-weight: 400\"><i><span style=\"font-weight: 400\">Complement Emulators with manual testing<\/span><\/i><span style=\"font-weight: 400\"><span style=\"font-weight: 400\">&#8211; Emulators come in handy for smart TV app testing as it is not possible to test on all versions of smart TVs as they are pretty expensive. However, some platforms like ROKU do not provide emulators and some existing emulators of platforms does not support testing of all necessary functionalities such as video streaming or network checking<\/span><\/span>&nbsp;<\/li>\n<li style=\"font-weight: 400\"><i><span style=\"font-weight: 400\">Network use cases<\/span><\/i><span style=\"font-weight: 400\"><span style=\"font-weight: 400\">&#8211; Sometimes testers might overlook scenarios where network connectivity is minimal. Under such circumstances, the app might freeze, close abruptly or display other irregularities. Testers need to test these use cases so that the right error messages are displayed in case of low network connectivity<\/span><\/span>&nbsp;<\/li>\n<li style=\"font-weight: 400\"><i><span style=\"font-weight: 400\">Resolution issues<\/span><\/i><span style=\"font-weight: 400\"><span style=\"font-weight: 400\">&#8211; Smart TV apps are built using multiple technologies. During Testing, it might so happen that an app may work fine on one smart TV with a particular resolution and freeze on another as smart TVs do not support pixel CSS<\/span><\/span>&nbsp;<\/li>\n<li style=\"font-weight: 400\"><i><span style=\"font-weight: 400\">UI issues<\/span><\/i><span style=\"font-weight: 400\"><span style=\"font-weight: 400\">&#8211; It is much more than cosmetic fixing. For smart TV apps, <a title=\"The What and How of GUI Testing\" href=\"http:\/\/www.tothenew.com\/blog\/the-what-and-how-of-gui-testing\/\">UI testing<\/a> is essential. While testing on different platforms, design issues are bound to crop up. Smart TV apps should have minimalistic feature sets with minimal user steps<\/span><\/span>&nbsp;<\/li>\n<li style=\"font-weight: 400\"><i><span style=\"font-weight: 400\">Usability<\/span><\/i><span style=\"font-weight: 400\">&#8211; Testers need to look into the seamlessness of the app experience from the end-user perspective. Developers and testers should focus on keeping the app experience minimalistic with single click functionalities, which means a minimal number of buttons on the remote. Navigation should be simple and easily comprehensible, visual rendering should be of high quality, sharp (alignment and spacing should be taken into consideration) and should adjust as per the screen size available and resolution should be optimal. Loading and buffering should be indicated by the loading indicator and the search capability should allow enriched browsing and content discovery.<\/span><\/li>\n<\/ul>\n<p>&#8211; Navigation movements should be predictable<\/p>\n<p>&#8211; Limit screen transitions<\/p>\n<p>&#8211; App should not require a separate guide<\/p>\n<p>&#8211; Keep\u00a0the focus clear including user\u2019s existing on-screen and selected component<\/p>\n<h3><span style=\"font-weight: 400\"><strong>Case Highlight:<\/strong>\u00a0\u00a0<\/span><b>Smart TV Application Testing for Jadoo TV\u00a0 \u00a0 \u00a0<i>\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 <\/i><\/b><\/h3>\n<p><span style=\"font-weight: 400\">TO THE NEW has expertise in<\/span> <a title=\"Smart TV apps development\" href=\"http:\/\/www.tothenew.com\/blog\/top-6-key-considerations-when-developing-smart-tv-application\/\"><span style=\"font-weight: 400\">Smart TV application development<\/span><\/a><span style=\"font-weight: 400\"> for Android TV,\u00a0<\/span><span style=\"font-weight: 400\">Apple<\/span><span style=\"font-weight: 400\"> TV, Amazon Fire, and<\/span> <a title=\"Roku TV app development services\" href=\"http:\/\/www.tothenew.com\/blog\/things-to-be-taken-care-while-testing-roku-tv\/\"><span style=\"font-weight: 400\">Roku TV<\/span><\/a><span style=\"font-weight: 400\"> as well as on Samsung Native, Samsung Tizen platforms. Recently, we developed Jadoo TV&#8217;s smart tv applications for 7 different platforms.<\/span><\/p>\n<p>Jadoo TV delivers Live and On-Demand content to viewers via its proprietary Internet based set-top box (STB). The client wanted to take its services to Smart TV viewers for higher engagement.<\/p>\n<p><span style=\"font-weight: 400\">We custom developed a highly engaging and intuitive app for seven platforms including Tizen, Samsung Orsay, LG, Roku TV, Android TV, Amazon Fire and Apple TV. We used platform specific languages such as <a href=\"http:\/\/www.tothenew.com\/java-development-services\">Java<\/a>, HTML, Brightscript, and Swift followed by rigorous testing before going live with these applications on respective stores.<\/span><\/p>\n<p>Over 500 test cases were created, and rigorous ongoing testing of applications was done before pushing it to the app store to reduce commercial and technical risk-<\/p>\n<ul>\n<li><span style=\"font-weight: 400\">Each platform-specific guidelines were met. Developer, store and QA checklist were checked thoroughly before submission in store<\/span><\/li>\n<li><span style=\"font-weight: 400\">Exhaustive test cases developed for all platforms and complexities were defined. Installation and uninstallation of app, splash validation, TV form factor support, UI design, navigation, testing of menu buttons, focus and remote functionality to validate gesture were carried out in detail<\/span><\/li>\n<li><span style=\"font-weight: 400\">Performance testing was done under differing network scenarios; load testing and API integration testing was conducted after each build. Smart TV apps do not come with dedicated CPU and RAM for processing. Hence it is important to check the app responsiveness and put in place load indicator in case it takes more than 3 seconds to respond<\/span><\/li>\n<li><span style=\"font-weight: 400\">The team focussed on tests that are easily overlooked. They made sure there was no conflict in focus and navigation, undertook memory optimization, thoroughly checked remote functionality and keyboard handling, ensured smooth scrolling, validated localization (for Roku app) to see if channel is restricted to a particular region, checked in-app video streaming, privacy functionality for user opt-in, validate load time and other such critical functionalities<\/span><\/li>\n<\/ul>\n<h2><b>Takeaway<\/b><\/h2>\n<p><span style=\"font-weight: 400\">Smart TV app market is growing rapidly. Apple, Android, Roku, Tizen, LG, Opera and others are coming up with new features every passing day and their quality checklist is getting rigorous. Brands looking to enter the smart TV app market needs to focus on delivering a quality app to the app stores. Hence, developing multiple test case scenarios are of primary importance. Monkey testing does not investigate all test case scenarios as only certain test cases are picked up randomly by the testing team. If even a small bug is missed, it can compromise the end-user experience. Some of the companies offering <\/span><a title=\"Smart TV app development services\" href=\"http:\/\/www.tothenew.com\/mobile-smart-tv-application-development\">Smart TV application development service<\/a> <span style=\"font-weight: 400\">also invest in Smart TV lab so that the testing is seamless and all the bugs are fixed across platforms and TVs. As there is a lot of effort in developing Smart TV app, it is imperative to go through platform guidelines thoroughly for avoiding future rejections and rework. <\/span><\/p>\n<p>If you have some more insights and interesting use cases on smart tv application testing, please feel to share them with us.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The pattern of content consumption has changed markedly in the last two decade. From mobile phones in the 90\u2019s to smart phones to smart TVs, the demand for smart connected devices continues to grow. According to a Gartner consumer survey, worldwide, by 2018, 87 percent of the TVs shipped annually will be smart TVs. The [&hellip;]<\/p>\n","protected":false},"author":1192,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":16},"categories":[1818,3479,1817,3477,1,1816],"tags":[4326,4722,1561,3982,3983,3981,4524,4483,3632],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/51610"}],"collection":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/users\/1192"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/comments?post=51610"}],"version-history":[{"count":13,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/51610\/revisions"}],"predecessor-version":[{"id":52064,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/51610\/revisions\/52064"}],"wp:attachment":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/media?parent=51610"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/categories?post=51610"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/tags?post=51610"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}