Epic Lanka Technologies, the company responsible for digitalizing the documentation and workflow in the National Medicines Regulatory Authority (NMRA), has offered it several options to recover the attachments that have been lost as a result of files being deleted from the Lanka Government Cloud (LGC).
One of the options the company is offering is for vendors whose applications are ongoing to be able to resubmit them. Around 6000 applications from some 150 vendors were in the pipeline when the files were deleted.
The company was also about to start a forensics recovery process with two external teams and one internal team but the work could not get underway because of a Court Order prohibiting them from accessing the system.
The deletion of the files come at a time there is a shortage of medicines in the country and there are attempts to link the two. It is just under two months since the deletions took place and the process of importing medicines takes at least four months. The implications will be felt several months later if at all. There should be no reason for work in the NMRA to stall because of the loss of files. Except for the attachments in the 6000 applications the rest of its data is intact. Medical experts point out that the renewal of licenses is something which can be done manually and the way it used to be before digitalization.
‘We are trying to mitigate the damage as much as possible’ said Dr Nayana Dehigama, the Chairman of Epic. ‘The option to resubmit the applications which can be done via a special window in the system will restore the files in two – four days and is the best because the forensics process does not guarantee the retrieval of all the files’.
On 9 July, Epic was informed by the NMRA that the PDF and JPEG attachments in applications could not be seen in the system anymore. Usually, these applications are for the approval and renewals of pharmaceuticals and devices which are imported and purchasedlocally. When Epic followed up on the online complaint, they traced the cause to human errorand found that the relevant attachments had been deleted from the file server during routine maintenance which is carried out by them. Attachments, because they are heavy documents, are stored in a separate file server in the LGC while the data in the main database is also stored in the cloud, but separately. The company hosted the files in the LGC in compliance with their contractual obligations.
According to Epic, the deletion of attachments did not occur because of a malfunction, breakdown or a fault of the eNMRA document management and workflow application software system.
‘We acted swiftly to recover the missing data and also immediately brought it to the notice of NMRA and Information and Communication Technology Agency (ICTA)’, points out DrDehigama. ICTA was the consultant to the NMRA and managed the digitalization project for them.
‘After conducting a comprehensive assessment of the system we informed the NMRA of the total extent of the impact on it, which is limited to the loss of supporting documents namely attachments, which are in that folder. As per our investigations there has been no data breach’.
Epic also confirmed that the data in the main database of the system that contains all information and which is fed into the system (relating to medicines and device registrations, licenses, approvals, renewals) are in-tact and has not been affected’.
The eNMRA system had no Disaster Recovery Arrangement in place. The files could have been retrieved in one day if such an arrangement had been provided. Although ICTA and the NMRA had several rounds of talks to consult EPIC on disaster recovery, they had not gone beyond February or March this year. It was around this time that the NMRA was going through an upheaval and the resignation of several of its Board members.
‘In the interests of transparency and to make sure there were no external or internal influences, as the chairman of the company Itook the decision to inform the CID and a complaint was lodged with them on 23 July. It was the CID which took the Court Order’. The company meanwhile has carried out its own internal inquiry and has handed over the report to the CID.
‘The reputation of the company is crucial’, points out Dr Dehigama. ‘This is why we have taken these measures proactively. We are in touch with both the NMRA who is our client and ICTA to start recovering the files but their response is that they have not got clearance from the CID so far’.
This is the first time that Epic has encountered this kind of system failure. In its 23 years of service the company has implemented over 100 large-scale ICT solutions including mission-critical systems, in both private and public sector organizations both in Sri Lanka and South East Asia. ‘To give you an idea of scale, we have about 14 banks on our client list and their transactions run into trillions of rupeesevery month and there have been no glitches’.
The digitalization of the document and workflow system in the NMRA was to streamline and make the process operationally efficient and reduce potential malpractice to bring greater transparency to its work. It was a move which was met with resistance both internally and externally. Prior to digitalization, applications were submitted manually and could take as long as two years for completion.
Preliminary preparatory work on the digitalization was carried out by ICTA and the NMRA. In 2018 after a stringent evaluation process by them the contract to develop the eNMRA document and workflow management system was awarded to Epic through a competitive tender. Before developing the product which had to meet the specific needs of its client, EPIC completed a SRS or Software Requirements Specification. This was based on the Framework of Requirement which was provided by the NMRA. All three parties were involved in coming up with the SRS which took several months and several rounds of discussions and even a residential before it was completed. Work on developing the software commenced only after the SRS was signed off by all the parties. The software was launched in April 2018 after a User Acceptance Test was carried out to confirm that the product complied with all the features in the SRS. Epic has a service contract for which they are paid a monthly fee by the government. All the users in the NMRA have access to the files while the company has a system route access which was mandatory with its service contract. Epic has assigned a team of four to specifically oversee the NMRA system which is supervised by an experienced team lead and Linux certified maintenance engineer. Following a decision by the NMRA Board towards the end of 2020, a team of experts from the engineering faculty of the University of Moratuwa carried out an integrity test on the system in February this year.
The eNMRA system, which was the first its kind, was so successful that the WHO viewed it as one of the best and most progressive systems and wanted it adopted in the region. While it allowed users to submit their applications and make inquiries online, because of its ability to make both online or offline payments it significantly enhanced the flow of information. One of the advantages of it is that vendors can send in their applications from anywhere in the country. In addition to operational efficiency the system allows workflow activities to be tracked and keeps records of them as audit trails which has strengthened the integrity of the NMRA’s processes.
