The Research Database Complex (RDC)
On this page:
- System overview
- System information
- System access
- Computing environment
- Transferring your files to the RDC
- Application development
The Indiana University Research Database Complex (
rdc.uits.iu.edu) supports research-related databases and data-intensive applications that require databases. The RDC supports Oracle and MySQL databases, and provides an environment (
rdcweb.uits.iu.edu) for database-driven web applications focusing on research.
The system runs Red Hat Enterprise Linux 5. User home directories reside on a Network-Attached Storage (NAS) device, with disk storage of 10 GB per user. This space is shared by your Big Red, Quarry, and Mason accounts, if you have accounts on those systems.
The RDC currently supports several important research projects:
- Collaborative Initiative on Fetal Alcohol Spectrum Disorders
- Indiana Spatial Data Service
- National Gene Vector Laboratories
Note: The RDC is strictly devoted to supporting research; it is not an instructional or classroom environment. If you need to use Oracle or Microsoft SQL Server in an instructional environment see Instructional database and web server accounts.
The RDC offers Oracle 11g Release 2 (version 22.214.171.124) and MySQL Enterprise Server (version 5.5.8 Advanced) database accounts, with a full suite of Oracle components that support:
Content: Oracle components on the RDC that support content include:
Advanced Security Option (ASO): Provides data
encryption and strong authentication services to the Oracle
Application Express: Offers development and
deployment of secure applications through a rapid, web-application
development tool for use with Oracle databases
Large objects (LOBs): Lets you store and
manipulate large blocks of unstructured data, such as text, graphic
images, video clips, and sound waveforms, in binary or
Oracle Multimedia (formerly Oracle interMedia):
Provides a platform for a wide range of multimedia-intensive
Oracle Text: Indexes any document or textual
content to add fast, accurate retrieval of information to Internet
content management applications, e-business catalogs, news services,
and job postings; indexes content stored in file systems, databases,
or on the web
Oracle XML Database: Treats XML as a
native datatype in the database
Oracle XDK: Contains the basic building blocks
for reading, manipulating, transforming, and viewing XML documents,
whether on a file system or stored in a database
- Partitioning: Lets you split large tables and indexes into smaller, manageable components, without requiring changes to underlying applications
Analysis The RDC provides these Oracle components that support analysis:
Oracle Data Mining: Provides a way to access
information buried in the data by creating models to find hidden
patterns in large, complex collections of data; embeds data mining
within the Oracle database; algorithms operate natively on relational
tables or views, eliminating the need to extract and transfer data
into other tools, applications, or servers
- OLAP: Offers in-database, advanced multidimensional analytic capabilities
Java: The RDC offers these Oracle components that support Java:
JServer Java Virtual Machine: A Java Virtual
Machine (VM) that runs within the Oracle database server's address
- Oracle Database Java Packages: Classes for relational database management system (RDBMS) features
|System configuration||Aggregate information||Per-node information (when applicable)|
|Machine type||Research database system|
|Operating system||Red Hat Enterprise Linux 5|
|Memory model||Distributed and shared|
|CPUs||Intel Xeon E5620 2.40 GHz (HP)
Intel Xeon Quad Core 1.6 GHz (Dell)
|Nodes||3 Hewlett Packard DL 180 G6 Oracle servers
1 Hewlett Packard DL 180 G6 MySQL server
1 Dell 2950 Database Driven Web Services
|RAM||288 GB||72 GB (HP)
8 GB (Dell)
|Local storage||Hewlett Packard StorageWorks 2000fc Modular Smart Array, approximately 48 TB of usable storage|
|RPeak||307.2 gigaflops||76.8 gigaflops|
|Storage information||Aggregate information||Per node|
|File systems||RDC home directory disk space is allocated on a Network-Attached Storage (NAS) storage device.
|Total disk space||48 TB|
|Total scratch space||
|Quotas||15 MB per user|
|Backup and purge policies||Incremental backups of the RDC Oracle databases occur at various
times between 1am and 6am, Sunday through Friday.
Full backup occurs between 1am and 5am every Saturday. Backups are retained for 30 days.
Backups for MySQL database servers on the RDC are the responsibility of the user.
|Availability scope||Access to the RDC is available to all IU graduate students, faculty, and staff. Undergraduates and non-IU collaborators must have IU faculty sponsors.|
Note: Indiana University will soon replace its current Data Capacitor with Data Capacitor II, a high-speed, high-capacity storage facility for very large data sets. With 5 PB of storage, Data Capacitor II will support big data applications used in computational research. IU partnered with DataDirect Networks, Inc. (DDN) to develop Data Capacitor II, which is scheduled to be installed in the IU Data Center in spring 2013. For more about Data Capacitor II, see the November 8, 2012, press release. If you have questions about how the change to Data Capacitor II will affect your research, email the High Performance File Systems group.
Requesting an account
Access to the RDC is available to all IU graduate students, faculty, and staff. Undergraduates and non-IU collaborators must have IU faculty sponsors. To request an account:
IU researchers: Use the Account Management
Service (AMS); see Instructions for getting more computing accounts at IU
- Non-IU researchers: Get a sponsored affiliate account (see Sponsored computing accounts for IU affiliates). Then, create your RDC account (see Getting your first computing accounts at IU).
Note: If you don't see RDC as an option in AMS, email IU Account Administration.
You will receive a confirmation email message once your RDC account is created.
Your database login
When you receive the email message confirming your RDC account is created, you must complete the process by requesting a database login. The confirmation email message will direct you to the online RDC Database and Web Services Account Application.
Database group accounts, where a username is shared by more than one researcher, are available on the RDC. You can request a database group account on the RDC and Web Services Account Application. To request a database group account, your group must have an existing IU Network ID, and you must provide the Network ID usernames of everyone who will be using the database group account. Whoever requests the group database account will be considered the responsible party for the account, and is responsible for communicating with the group database account's users regarding system downtime and other information. For more about IU group accounts, see Requesting a departmental or group account
When your database login has been created, you will receive another confirmation email message containing your login credentials, and information about connecting to your database.
To request an Oracle or MySQL database, email the UITS High Performance Systems (HPS) team. After the HPS team creates your database, you will receive a welcome message in email containing information about your database username and password.
Note: If you already have an Oracle or MySQL database login, but don't remember the database password, refer to your database welcome message. If you need help, email HPS.
Connecting to your Oracle or MySQL database
Oracle: For instructions on connecting to your Oracle database on the RDC, see:
- On the Research Database Complex at IU, how do I access my Oracle or MySQL database?
- How do I access my database on the IU Research Database Complex using the Oracle client for Windows XP?
- How do I access my Oracle database on the IU Research Database complex using Aqua Data Studio?
- In Windows, how do I set up an ODBC source to access my Oracle database on the IU Research Database Complex?
MySQL: For instructions on connecting to your Oracle database on the RDC, see:
- On the Research Database Complex at IU, how do I access my Oracle or MySQL database?
- At IU, how do I use the phpMyAdmin web interface to administer my MySQL database on the Research Database Complex?
- On the RDC at IU, how do I stop or start my MySQL database?
The shell is the primary method of interacting with the RDC. The command line interface provided by the shell lets users run built-in commands, utilities installed on the system, and even short ad hoc programs.
The RDC supports the Bourne-again (
tcsh), C (
csh), Korn (
and Bourne (
sh) shells. New user accounts are assigned
bash shell by default. For more on
see the Bash
Reference Manual and the Bash (Unix
shell) Wikipedia page.
To change your shell on the RDC, use the
chsh (instead of
changeshell) changes your shell only on the node on which
you run it, and leaves the other nodes of the cluster unchanged;
changeshell prompts you with the shells available on the
system, and changes your login shell system-wide within 15
Environment variables: The shell uses environment variables primarily to modify shell behavior and the operation of certain commands. A good example is the PATH variable.
When the shell parses a command you have entered (i.e., after you
Return), it interprets certain
words you've typed as program files that should be executed. The shell
then searches various directories on the system to locate these
files. The PATH variable determines which directories are searched,
and the order in which they are searched. In the
shell, the PATH variable is a string of directories separated by
/bin:/usr/bin:/usr/local/bin). The shell
searches for an executable file in the
/usr/bin directory, and finally the
/usr/local/bin directory. If files of the same name
foo) exist in all three directories,
/bin/foo will be run, because the shell will find it
bash shell, use
echo to display
the value of an environment variable:
To change the value of an environment variable:
Startup scripts: Shells offer much flexibility in
terms of startup configuration. On login,
bash by default
reads and executes commands from the following directories (and in
~ (tilde) represents your
home directory (e.g.,
~/.bash_profile is the
.bash_profile file in your home directory).
On logout, the shell reads and executes
~/.bash_logout. For more on
files, see the "Bash
Startup Files" section of the Bash Reference Manual.
Transferring your files to the RDC
The RDC supports SCP and SFTP for transferring files. SCP is a command line utility included with OpenSSH. Basic use is:scp [[user@]host1:]file1 [[user@]host2:]file2
For example, to copy
foo.txt from the current
directory on your computer to your home directory on the RDC, use
username with your Network ID
scp foo.txt email@example.com:foo.txt
You may specify absolute paths or paths relative to your home directory:scp foo.txt firstname.lastname@example.org:some/path/for/data/foo.txt
You also may leave the destination filename unspecified, in which case it will become the same as the source filename. For more, see In Unix, how do I use SCP to securely transfer files between two computers?
The SSH File Transfer Protocol (SFTP) provides file access, transfer, and management, and offers client functionality much like FTP. For example, from a computer with a command line SFTP client (e.g., a Linux or Mac OS X workstation), you could transfer files as follows:$ sftp email@example.com firstname.lastname@example.org's password: Connected to rdc.uits.iu.edu. sftp> ls -l -rw------- 1 username group 113 May 19 2011 loadit.pbs.e897 -rw------- 1 username group 695 May 19 2011 loadit.pbs.o897 -rw-r--r-- 1 username group 693 May 19 2011 local_limits sftp> put foo.txt Uploading foo.txt to /N/hd02/username/RDC/foo.txt foo.txt 100% 43MB 76.9KB/s 09:39 sftp> exit $
Graphical SFTP clients are also available for many systems. For more, see Transferring files with SFTP
Web Services: In addition to providing a home for
research databases, the RDC provides an environment for
database-driven web applications with a research focus. This
environment is composed of a Dell 2950 with a 1.6 GHz Quad-core Intel
Xeon processor and 8 GB of memory. This system
rdcweb.uits.iu.edu) runs Red Hat Enterprise Linux
5. User home directories reside on the IBM N5500 NAS storage device,
with disk quotas of 10 GB per user. For details, see Web Services on the IU Research Database Complex.
Oracle: For further documentation, see the Oracle Database Documentation Library, 11g Release 2 (11.2), and the following online guides:
- New Features Guide
- Master Glossary
- Application Developer's Guide - Fundamentals
- SQL Reference
- PL/SQL User's Guide and Reference
- Application Express (ApEx) Documentation
- ApEx Developer's Guide
- ApEx Application Builder User's Guide
MySQL: For further documentation, see the MySQL Reference Manual.
Access to the RDC is provided to all IU graduate students, faculty, and staff. Access is also provided to undergraduate students and non-IU collaborators, if they have IU faculty sponsors. For information about user responsibilities and security issues, see Research Database Complex (RDC) usage policies.
The RDC is strictly devoted to supporting research. The RDC is not an instructional, classroom environment. If you are not doing research and wish to use a database, such as Oracle or Microsoft SQL Server, see Database and web server access for instruction.
Accounts remain valid only while the account holder is a registered IU student, or an IU faculty or staff member. On Big Red, Quarry, and the RDC, accounts are disabled during the semester following the account holder's departure from IU, and then are purged within six months. To request that your research systems account be exempt from disabling, email IU Account Administration. If the request is approved, the account will remain activated for one calendar year beyond the user's departure from IU, and then, at the end of the year, the account will be purged. Extensions beyond one year for research accounts are granted only for accounts involved in funded research and having an IU faculty sponsor, or with approval of the Dean or Director of Research and Academic Computing.
By submitting the RDC and Web Services Account Application, you affirm that:
- You understand use of the database is reserved for research purposes only.
- You will acknowledge use of IU's high-performance systems in publications resulting from your research.
- You will provide periodic listings of citations of those publications upon request.
Database group accounts
To request a database group account, your group must have an existing IU Network ID, and you must provide the Network ID usernames of everyone who will be using the database group account. Whoever requests the group database account will be considered the responsible party for the account, and is responsible for communicating with the group database account's users regarding system downtime and other information. For more about IU group accounts, see Requesting a departmental or group account
As owner of a database account, you are also responsible for:
- Creating and managing your schema objects (e.g., tables, views, procedures, triggers, and schema privileges)
- Changing datatypes
- Any data processes, such as data imports, deletes, modifications, transformations, and retrievals
- Creating and maintaining copies of scripts
- Emailing the HPS group about changes in space or database administration, or if you no longer need access to the research database
- Adapting your schema and data as required during system and database upgrades
- Adapting client applications and tools to system and database versions
- Monitoring HIPAA required audit logs, if auditing is enabled
If you need help with the above, submit a request for RDC database consulting services by emailing the HPS group.
RDC database administrators are responsible for:
- Backing up the database
- Managing space allocation
- Managing database and tablespace creation
- Monitoring and reporting database performance
- Monitoring and reporting invalid schema objects
- Installing database and system upgrades and patches
Many of the technology services provided by the UITS Advanced Biomedical IT Core, Research Technologies, and Enterprise Infrastructure divisions are formally aligned with the federal Health Information Portability and Accountability Act (HIPAA). See About IU's research systems and services and HIPAA alignment.
Once your RDC Database and Web Services Account Application is processed, you will receive a confirmation email message that describes how to access your Oracle database, and how to reset your initial database password. Your initial database password will be sent in a second email message.
It is important to employ methods that do not transmit passwords across the Internet in plain-text format. If you use SQL*Plus to access an Oracle database, invoke it without the password. The following example shows how to connect to Oracle fromdoe@RDC:~> sqlplus email@example.com
Provide the password when prompted:SQL*Plus: Release 10.2.0.1.0 - Production on Fri Mar 14 13:49:03 2008 Copyright (c) 1982, 2005, Oracle. All rights reserved. Enter password:
To connect to MySQL fromdoe@RDC: $mysql --defaults-file=/N/u/<username>/RDC/.my.cnf -u root -p
usernamewith your username. When prompted, enter your password.
Database backup and recovery
UITS performs incremental backups of the RDC Oracle databases at various times between 1am and 6am, Sunday through Friday, depending on the instance. A full backup occurs between 1am and 5am every Saturday. Backups are retained for 30 days. In the event of system failure, research databases can be restored to the point of the last good backup, which is usually from that morning. Data recovery for individual accounts is not guaranteed if data loss is the result of user error.
Recovery of a table is typically not part of the
database recovery process. Dropped tables can often be recovered
recyclebin feature; see In Oracle 10g and later, how do I recover a dropped database table?
Backups for MySQL database servers on the RDC are the responsibility of the user. Use the following command:mysql_instance backup
For more about using the
mysql_instance command, see
On the RDC at IU, how do I stop or start my MySQL database?
Scheduled down time
The RDC maintenance window is the first Tuesday of each month, 8am-5pm. Notice of any emergency downtime will be posted at IT Notices.
Disk space for data loading and other applications
If you need space on the RDC for staging data, or for data-related packages and applications, email the HPS group, which will evaluate requests on a case-by-case basis.
Production mail service is not provided on the RDC.
Support staff are available from approximately 8am-5pm Monday-Friday. Email the HPS group to report problems with the RDC. Be sure to include:
- The name of the database server to which you were connecting
- A description of what you were trying to do
- A description of the problem
- The name and version of the tool you were using to connect to the Oracle database server
- Your computer's operating system
- The error message number and text, if applicable
- The time the problem occurred
The UITS High Performance Systems (HPS) group supports and administers the RDC; if you have questions about the system, email HPS. If you have questions about compilers, programming, scientific or numerical libraries, or debuggers, email UITS Scientific Applications and Performance Tuning. If you have questions about statistical and mathematical software, email Research Analytics.
This is document amuw in domain all.
Last modified on March 14, 2013.
Last modified on March 14, 2013.