We all remember too well the cases of Enron and WorldCom. In light of those cases, the government has created new laws commonly known as the Sarbanes-Oxley (SOX) Act of 2002 to protect investors and insure the accuracy of corporate disclosures. SOX-type regulations are being implemented around the world, including in Information Technology organizations. These regulations revolve around setting up internal controls on many aspects of record keeping and being able to verify those controls as well. Traditionally, the database administrator (DBA) has access to all the corporate data, so that he or she can perform their assigned tasks, such as data import and export, creating reports, and maintaining database performance and stability. Such a wide and deep level of exposure to the data is typically given to DBAs; however, DBAs have limited tools to provide the levels of controls and types of reporting for addressing both internal and external threats that are being expected of them under these new regulations.

This article will deal more specifically with insider threats to IT, which today are seen as much greater than outside threats. We will describe our research and findings and describe how the IT professionals we interviewed are implementing the necessary products, policies, and procedures to reduce insider threats and provide the necessary reporting for regulatory compliance.

The regulators are here

While the most notable regulations in the US are SOX and the Health Insurance Portability and Accountability Act (HIPAA), there are similar regulations around the world.

Japan has the Financial Instruments and Exchange Law, which is commonly referred to in the US as J-SOX. Another regulation is Basel II: International Convergence of Capital Measurement and Capital Standards is a set of regulations set up by ten countries.

Depending on your industry, you may fall under one or more of these regulations. They are becoming the model for corporate governance across the globe in both public and private companies and will affect you even if you don’t directly fall under their regulations today. Table 1-1 shows some of the regulations and the threats they cover.

With all these regulations you would think what needs to be done is crystal clear, nothing could be further from the truth. Most of the regulations leave enough room for interpretation that you could drive a mainframe through them. In other words, many decisions are left to the IT organizations themselves, with serious ramifications to such decisions and those that make them.

Where are we now?

Currently most database vendors provide for security control and auditing of their RDBMS to a point. These include setting up groups, access controls to the column level and control of stored procedure execution, among others. Audit controls can be turned on at multiple levels. Basic level auditing might indicate whether a user was successful or unsuccessful at a login attempt. Detailed auditing could be at the level of each SQL statement. Auditing controls can also be turned on at the user level or the object level for additional detail. Such records are a great example of access to lots of data but not much information. Just turning auditing on is not enough because the records need to be processed in a way that extracts the key events you are concerned with, i.e., those events that might impact regulatory issues.

Auditing must be able to provide the necessary records to comply with SOX, J-SOX, Basel II, and in the medical case, HIPPA requirements. Auditing tools included with the most popular database vendors provide the information needed, but not without a cost. You must make sure you have sufficient CPU, disk, and memory resources available on the machine. In one example from our interviews, a client produced 30-50 gigabytes of log files per day. This level of logging clearly requires sufficient available resources to maintain the database performance at the necessary level.

However, there are a lack of tools that provide trend analysis, which could indicate a significant internal threat. There are tools for most databases that will search the audit file for specific access violations and provide alerts. This is good to know but not sufficient for internal threats, which typically come from people authorized to use the database. Consider the case where a technical support person is authorized to look up customer records to do his job. On a tough day, he might look up 40 records per day. When he accesses 100,000 records in a day, this is probably an indication of data theft. This can only be detected by trend analysis.

Traditionally, in databases, DBAs operate above the security grid. This puts the DBA at risk because his or her actions can break the audit trail due to their access to every piece of data including log files. Such a wide and deep level of exposure to the data can make the database administrator a material witness (legally speaking) in related litigation. What is needed is a viable process to give the DBA a way to continue to do the job, but without seeing sensitive data such as financial or personal data (e.g., medical records – HIPPA).

So what can you do?

Given the set of tools generally available with most databases, what can you do without moving to products specifically targeted to solve these problems?

Most regulations are concerned with a few specific issues:

• Having effective internal controls
• Detecting unauthorized use
• Controlling and verifying access
• Enforcing system maintenance
• Providing information for independent audit

Having effective internal controls means you have reviewed your operation, have identified risk areas and have policies and procedures in place that mitigate the risk you have found. Auditors will be looking to see that you have made reasonable efforts to determine if there has been a compliance violation. These “reasonable efforts” include, but are not limited to:

• Having auditing turned on, audit log file secured and auditing records reviewed
• Be able to trace users accounts to an approved request for setup and make sure that password aging is turned on
• Having a change management system in place with approvals
• Tracking down time, because during unauthorized down time someone could be trying to manipulate the system
• Having a backup and disaster recovery plan to protect the data you will need for reporting

Enforcing system maintenance includes making sure that all appropriate patches are applied. Typically, security patches must be installed as soon as possible; of course the requirement for a system restart on some patches may require scheduling during an acceptable maintenance window. Other patches should be reviewed and evaluated as to when they need to be applied.

Providing for independent audit means that there are methods (hopefully easy ones because auditors charge for their work) where auditors can determine that the processes set up in building controls were actually followed. Example: your company’s policy is that a terminated employee’s access must be removed before the employee leaves the building at termination. If the auditor goes through the HR system and selects an employee, they should be able to look up the record showing database access was removed and match that to the time of termination.

As for protecting yourself as a DBA, there are few things you can do. For example, partition applications as much as possible with each partition controlled by a separate login and remove general system access. This way if a problem is found it will be localized. Make sure that actions you take can be traced back to some type of approval or system requesting the action, such as a change management system.

Here is what one of our UK interviewees set up with regard to controls.

• Centralized account creation for ease of auditing which entails:
• Definition of "Application Owners" to approve requests for access. These Application Owners have a complete understanding of the applications at a technical level so they understand what access should be required for various job roles. We use Oracle "workflow" to process the approvals, which has made things a lot more straightforward - approvers get sent an email and they can reply with their approval. All very painless.
• For application support staff, we define database "roles" which limit their access to the specific subset of tables that they need to access to perform their day-to-day work. We are now deploying a new Oracle utility called "Dataguard," which performs the same kind of access controls.
• Audit scripts run daily to confirm that all accounts that exist on the system are approved.
• For application accounts, we defined exactly which access (in Ebiz suite these are called "Profiles") staff need - all unnecessary access was revoked.
• All access - application and database - is reviewed every 3 months.
• All key system passwords are changed every 6 weeks.
• Access to databases and systems is limited to Technical support staff (DBAs and System Administrators).
• Auditing of logins and of the activities performed by the above accounts for technical staff.
• Auditing of access to key data at a database level.
• For database links, we have a dedicated schema on the "target" system for each remote system/application that connects in. Audit records for tracking what these remote systems are. The schemas are also named to identify their purpose. These schemas just have access to the tables that they need.
• We also have different levels of systems - development systems where development staff have access to all areas, Build Test, UAT [Unit Acceptance Test], Staging (production copy), and Production systems - changes have to be deployed upon these latter 4 types as "patches" - development staff do not have access. This way we get comprehensive testing of the patches and we keep the environments aligned in terms of what is where.
• For system changes, we have a tool in which scheduled changes are recorded. Depending upon the nature of the change, there is an approvals hierarchy:
• No approval required (minor changes, adding space etc.)
• Local manager approval required (areas that need technical review or project activities, but still nothing major).
• Change Control Board approval (significant system changes, need proof that they have been tried and tested on Build test and UAT environments).
• The system change tool doubles up as a utility to review all the changes that have been performed upon a system. Although, obviously people could make changes without using this utility, we have a lot of auditing in place to flag when changes have been made that are not in here. Staff now uses it as a matter of routine. All of our other utilities update these records as well, so they provide a great audit trail for everything that happens on an environment - handy when trying to identify why things aren't working.

Another interviewee from a medical company in the US southwest region said the following:

“Fortunately for us, our primary OE [order entry] and payment system, SAP and the VIRSA tool do an outstanding job of SOD [segregation of duties]. The IT department has procedures for handling new hires and terminations. The DBA group has developed automated procedures for password expirations and complexity. No user has the ability to make a direct connection between a data access tool and a database. Combinations of application security, LDAP integration or customized security roles are employed in all systems where financial transactions occur.”

As a recommendation he suggests the following:

• Pick a vendor with a good product that meets SOX certification before you buy it.
• Make sure they offer robust data management tools.
• Make sure those products have robust application level security and auditing features.
• Make sure you implement a security matrix and corporate-wide procedures for the granting of security.
• Identify business owners for security roles and require the owners’ signatures for granting security.
• Don't depend on Oracle DBMS security as the only line of defense.

Securing your systems

Some organizations will have whole groups dedicated to security. If yours in not one of those, here are some things you should think about.

Account setup and passwords were already mentioned. Next is how people access the system. Make sure that the authentication is done via a secure protocol depending on your access method, such as SSL. Additionally, consider encrypting the dataflow as well.

Another technology you can use is an application security gateway or database firewall. These are devices that understand the application and can track user access. One method of doing this is by deep packet inspection where each packet going over the network to the database server is examined to see what type of access is being attempted and determines the user. Application security gateways can provide other benefits such as virus checking, anomalous behavior detection and determination of multi-stage attacks. Some of these systems also have modules that provide compliance information specifically targeted to SOX and HIPPA. Unfortunately if you use certain types of encryption on the traffic going directly to the database then a firewall that uses deep packet inspection will be prevented from seeing inside the packets and therefore doing their analysis.

If you are using iSCSI protocol that puts your virtual disks on Ethernet, this is another way your data is exposed. It should be considered if there is a chance of someone using network “sniffer” or other means of unauthorized data access. If so, consider using something like IPSec to encrypt the data.

In both cases be aware because there may be performance implications; see the section on performance below for more information.

Products that can help

According to Forrester Research, database vendors and ISVs will be coming out with more tools to help DBAs with access control over the next 18-24 months. One of the existing tools today is Oracle Database Vault.

Oracle Database Vault helps in dealing with insider threats, addressing regulatory compliance demands, and establishing separation of duties. You can setup flexible security policies to address the needs of your enterprise. It can also be used to separate the DBA from the sensitive data. There is also Oracle Secure Backup. It is a tape backup management product that provides secure backup for Oracle database and operating system files. It stores data in encrypted format that allows protection in case of theft.

Another product on the market today is from IPLocks (http://www.iplocks.com). According to them, “the IPLocks Solution assesses the vulnerability of databases, proactively monitors data users, sessions and objects, and allows forensic auditing of logs. The IPLocks Database Security and Compliance Solution utilize processes and technology to reduce inherent risk to business critical data that can be misappropriated. IPLocks safeguards sensitive data while creating a unique information risk management solution for enterprise customers.”

Imperva (http://www.imperva.com) has products that fit into the application security gateway or database firewall category discussed above. SecureSphere monitors and protects sensitive information in databases by assessing, monitoring, and auditing all access to an organization’s databases, and it tracks and controls privileged user activity across the infrastructure. It does this by reconstructing the full request response pair at the database level through deep packet inspection. SecureSphere handles traffic that is encrypted on the network and it provides reporting to make compliance with the new laws discussed easier.”

Guardium (http://www.guardium.com) has their SQL Guard product line, which provides database protection and simplifies reporting for Sarbanes-Oxley, GLBA, HIPAA, and Basel II. SQL Guard is a network based appliance that intercepts all network traffic going to and from the database you want to monitor. SQL Guard has passive and database firewall modes. It provides a number of tools focus on what auditors want to see, including change control and automated compliance workflow.

Also, check with your auditors for tools they might have. As you would expect, since these are the folks who will be checking up on you, they typically have tools they use to help sniff out trouble. In some cases you may be able to get the tools for free; in others the auditor may sell the package. The other advantage in using the auditors’ tool is that it may reduce your auditing bill as it should be easier for them to perform an audit with the tool already installed.

Performance

System performance may fall victim if overlooked while doing database auditing or enabling security. If the system is not properly configured or lacks capacity, then actions such as SQL statement level auditing may cause significant performance degradation. One vendor we spoke to said that with an adequately sized system, enabling auditing should only impact system performance by about 3%. In the example mentioned previously, when a client produced 30-50 gigabytes of log files per day, CPU utilization was up by about 10%. Some people reported as much as 30% system overhead with Oracle auditing turned on. The most probable causes of this are either improperly set database parameters or a bottleneck in one or more of CPU, disk, or memory capacity.

Another performance issue to keep in mind is enabling security. When using security for authentication only, the performance impact should go almost unnoticed. However, encrypting every network packet may cost some CPU time. If this becomes a problem, then consider upgrading to a gigabit or higher speed network, enabling jumbo packet size, and getting a high end Ethernet adapter which can offload some of the packet processing.

In addition, new technologies such as iSCSI protocol should be carefully analyzed before putting them to use. It is very convenient to have a SAN where the disk array is not connected directly via SCSI cables to the server. However, the following items should be taken under consideration. Bandwidth -- does your installation have enough to handle OLTP or DSS system now and in the near future? What would the network utilization be like during the peak hours? Encryption overhead -- enabling security for all network traffic between the server and SAN using IPSec or similar protocol can have a significant impact on performance that is much higher than encrypting server to client traffic. This is due to all the additional data manipulation that takes place on the server, such as transaction log, temporary storage, and so on.

The bottom line here is to make sure that your systems are properly configured and that there is enough overall system capacity to handle auditing and security overhead.

Conclusion

Every organization will eventually be affected by the regulations discussed here; it seems simply a matter of time. Be proactive and begin to address these issues sooner than later and keep in mind that as a DBA you may have some legal exposure on this subject as well. First, determine what is required, and then establish what additionally makes sense for your business. You have choices; look at the various packaged products to cover some of your needs. As this is a hot area, more tools will be coming out. Alternatively, like our interviewees, work on creating your own set of tools and scripts. If you choose the go-it-yourself path, check with your auditors to see what tools they will make available or require. Doing this may reduce duplication and save you time and money. In addition, be aware of performance considerations for any solution you implement – test and retest.

As a DBA, you can reduce your risk of exposure by implementing a tool like Oracle Database Vault that helps to separate you from the data. Since the major work will be a one-time effort to set up, and it is a very specialized area, do not forget to consider getting an outside specialist involved. They can also save you time and money.

I hope you found this article useful and informative. Please feel free to send your comments and suggestions.

About the authors

Alex Polishchuk is the founder and president of Advanced Computer Consulting (www.advcomputerconsulting.com). Alex has over fifteen years of professional experience designing, developing, and implementing database applications in various industries and companies ranging from small to Fortune 50 corporations. Alex’s primary areas of expertise are in database security and performance optimization and tuning. He can be contacted at alex@advcomputerconsulting.com.

Michael Procopio is a Senior Product Manager at HP. He has over 25 years of experience in computer systems and networking. He has held positions in consulting, product management, technical marketing, training and IT management. Michael has been a speaker at numerous IT conferences and is a member of the IEEE. He can be reached at michael@mprocopio.com.

Additional Resources

http://www.bis.org/bcbs/history.htm

Back to DBAsupport.com