Interoperability Testing (IOT) in Telecommunication – Challenges and Opportunities

Sometime back I got a message in Linkedin from one of my contacts about telecom testing, more specifically on IOT testing.Since I was a IOT tester before I thought I can share some knowledge about good and bad of IOT testing.

This was the message

I am XYZ, I am a Technical lead at ABC Corp, I have to know about field testing, What short of issues one can face during field testing, What kind of testing is IOT. Could you please provide me some information on this. It would be better if you share any doc related to Field/IOT testing.

Thanks a lot.

This article will help those who want to start a career in Telecom IOT testing or who wants to change their career path to IOT test.

Before starting to discuss about pros and cons of IOT test we need to know what really is IOT test.

What is IOT Testing?

IOT stands for Interoperability Test. Like other kind of testing in IOT UEs or Mobile phones are tested against networks and vice-verse. IOT is carried out to verify whether mobile phones act according to requirements against specific networks as well as mobile network works properly with mobile phones.

There are different requirements put of mobile phones to make them usable and safe for consumer usage. Some of the requirements are related to 3GPP specifications other related to different regulators like FCC and some related to mobile features like usage of camera, touch screen, etc.

The part we are going to discuss is about Telecom part of the testing which deals with 3GPP specifications.

Interoperability testing is divided into mainly two types:

• Field Test
• Lab Test

Field test refers to the testing done on a commercial network or pilot network to verify mobile or network behaviours. There are different reasons field tests are carried out around different art of the world mainly:

• To verify that UE or mobile can operate on real commercial network with other UEs
• To debug certain issues in UEs like call drops, throughput issues, handover issues, etc
• To debug some operator issues like loss of coverage, performance issue, so operators can tweak or upgrade network.
• To test a newly deployed per-commercial network
• For benchmarking against other mobile phones.

Field test can be static or mobile. When doing static testing like throughput verification, field test can be done at a static location. But most of the time field test are done driving around a specific area.

When drive test are done it can be carried out for many reasons like for regression test which is done to run a test scope, or for benchmarking against other equipments or mobile phones or for debugging, etc. Drive test can also be done driving around a specific route, most of the time provided by the operators or driving random locations for exploratory testing.

NOTE: If you have car sickness than field test is not what you should be looking for. You need to be doing lot of mobility test if you choose this kind of testing.

Lab Test on the other hand is completely different kind of test where a tester needs to visit a network vendor lab or to an operator lab to carried out different type of tests. These tests are carried out for certifications like GCF, or test early features (testing LTE-Advanced before commercial release) or for debugging. Lab sessions are most of the time paid sessions means UE manufacturers have to pay network vendors for the lab time. So as a lab tester you need to make full use of the lab time and try to finish the test scope within the specific time period. If you are struck because of some specific reason like UE software is not working or network configurations are wrong or other issue make sure you inform the things to the test coordinator to solve the things as soon as possible.

Now that we know what is IOT let’s check some of the challenges one can face in IOT both in lab test and field test.

Challenges in Interoperability Testing

Before you prepare to change your career path to IOT testing you must make sure that you are one of those who likes to travel and you enjoy being in different part of the world in different time. If you are not who can stay many days away from home, or you does not like to visit new places, meet new people and enjoy different food, IOT is not for you. This is a job where you need to be adventures.

Once I remember walking long hours in a small town in France to get food, just food not my favourite cuisines. This was another public strike day in France and everything was close. And the strange thing is that sometimes you end up in some places where no one knows English and you need to use sign language to make them understand what you really want. But in the end of the trip you must feel that you enjoyed it. Otherwise after some trips you will feel bored and that’s the beginning of the end of you IOT career.

Another biggest challenge in IOT is to deal with travel bookings and receipts. Booking travel is most of the time done by agencies but you are the one who will suggest them what they will book. This way your company should not be spending too much money on travel and hotel expenditures.

Also make sure that you need to keep all the receipts from your trips to file for reimbursements if there are any.

Apart from logistics, security is one of the thing which every IOT travel need to keep in mind. I had some bad experiences in different parts of the world, but luckily nothing serious. But every time you travel make sure someone else from you family knows where you are and they should have details about your flight, hotel and taxi.

When booking taxis for test you should make sure that the taxi driver speaks your own language or he should be speaking English. This will save a lot of time and energy to make him understand where you want to go and other things.

If you are going to a lab make sure that you take all the details about the lab’s location, the contact person whom you are going to meet and what time he will be there at the office. This will help you a lot.

Preparations Before IOT trips

Whether you are going to a Lab or you are testing in the field make sure you have the scope of what you are going to do. Say you are going to test for one week you should have a plan what you are going to finish and how you are going to finish. Prepare the hardware and software which is going to be used on the first day of the test session. check your laptop, drivers and other test tools. Many times it happens that the tools work great on your office local network but when you start them in the field or in a different network they stop working. So test these scenarios before.

Here is a small list of thing you should check before travelling

• Check you have valid permit to travel to the country and work there.
• Make sure your tickets, hotel reservations are done. Call and check with the hotel about the reservation
• Call and book a taxi if required. Check their rates and make sure that the driver speaks English or a language you understand.
• Prepare hardware, software, laptop and print the test plans.
• Get all contacts and address if you are going to a lab.
• Get a map of the locations where you are going to do field test if there is any

Some times it’s better to take an extra laptop for backup. It happened with me once that my laptop crashed, but luckily I got a spare laptop with me. A spare laptop will definitely add some extra weight to your luggage but you never know when you are going to need it.

Testing and Log Analysis

Testing, log analysis and creating new error reports are the majority of the work you are going to do in IOT. Log analysis is different in different organizations. Most of them use their own proprietary tools for capturing log and analysis. For text log analysis TextAnalysisTool is one of the best editor out there. It’s free and works like charm.

Error reporting is another big job to do in daily work. Every companies use different bug trackers but the fundamental in every bug tracker is same.

• Check in bug tracker if the same issue is raised before or not. Same issue in different networks may be considered as different bugs
• Make sure the title of the bug report is perfect, not too ling and with good to understand information.
• Write the detail descriptions with good amount of information. Like Where it is tested, what software versions used, what scenario. Some times GPS location of the exact spot helps to reproduce the issue. Put logs extracts which shows the issue.
• Fill field like radio technology used (GSM/WCDMA/LTE), SIM card type and other problem areas correctly. These small information helps a lot when fixing the issue.

There are many way to write a bad bug report but there is just one way to write a good bug report. Follow the user manual or technical manual provided to report errors, if still confused ask a colleague, that helps a lot.

Sometime you need to write test report in the end of a test session. This is mostly applicable to Lab test sessions. But many companies also encourage to write a small test report for field test. In the report cover briefly what is tested, what is left and what new bug reports are created. This will be useful for project reporting and can be used when planning next test sessions.

Getting Jobs in Interoperability Testing

Now that you are ready to start your career in IOT, make sure you update your CV and apply for the jobs. Like any other jobs, wireless engineers and specifically all mobile communication test engineers need knowledge on 3GPP specifications. Depending on what you are going to test the specifications change. But here are some of the most important 3GPP specifications which you must know before starting to apply for 3GPP IOT engineer.

• RRC Specification: This is one of biggest and important pacification. There are two variations of this specification, one is for UMTS technology which you can read in 25.331 specification and another is for LTE which is described in 36.331 specification.
• NAS Specification: For getting knowledge on layers like Mobility Management, GPRS Mobility Management, SMS, Session Management, etc follow Mobile radio interface Layer 3 specification 24.008

The easiest way to start reading about 3GPP is to start reading a book first.Books gives some basic overview about the technology and different sub areas in 3GPP technologies. Pick a book on the area you are going to work. This is a comprehensive list of books which you can follow for better understanding about UMTS and LTE technology.

Recommended UMTS and LTE Books

• WCDMA Design Handbook: WCDMA Design Handbook contains a lot of details about UMTS and many advanced topics are explained with good examples. WCDMA Design Handbook is a must if you are working on UMTS testing or development
• 4G: LTE/LTE-Advanced for Mobile Broadband: This book is very good for LTE and LTE Advanced Study.

Hope this article will help you in taking correct decision while starting or changing career to interoperability testing. Do you have some questions or want to give some feedback, post them in the comment section.