Guide to using the Custom Build Tool
A detailed user guide on how to user the Schemation DEM’s Custom Build Process in your development environment
The Schemation Database Environment Manager (DEM) Custom Build process is used to build database environments from source files held on a operating system. This is the user guide for that process. It shows how to define source code and source data in the manner expected by the DEM. It shows how to define the security set-up of your database design and how to invoke and run the Custom Build process.
What Schemation’s DEM DOSN'T DO
Schemation works currently only with Oracle database servers. It does not yet cover SQL*Server or any other database product. It does not provide a mechanism to install the Oracle executables themselves nor to create databases from scratch. It does not provide a mechanism for copying database images.
What Schemation’s DEM DOES
The Schemation DEM centralises, simplifies and eases the process of managing the creation, build and maintenance of multiple development environments. The DEM provides a way of translating flat file definitions for stored procedures, tables, indexes, data and security configuration information into working database schema groups (environments). This, build information, can be loaded into an Oracle Database in the DEM's own schema, from which it may be built into any number of different instances of the same (or modified environments.
The Schemation DEM works well on large projects, where there are a large numbers of developers, but, it also works well with smaller projects. The DEM records all build operations and presents them through drill down reports which may be viewed with a Web browser. The DEM typically provides support for whole teams of developers, each of which can have a separate development environment to work within.
Most build related 'administrative' work is performed through Schemation's web-based interface. This allows the DBA to create users or register users to work with the DEM. Registered users can be given controlling rights over a number of environments. Having controlling rights means that the user is able to build an environment either from the operating system or from the web. The DEM is able to reduce the burden on DBAs by providing users with the 'controlled' capability to drop and re-create their environments - without giving them any DBA privileges. All DBA tasks involved in building and rebuilding environments are performed by the DEM.
Types of Build
A user may request a build through the web-based interface (a Web Build), or through the operating system (a Custom Build). The DBA would typically define another type of build (called a Repository Build).
Custom Build
A build requested through the operating system would normally be used where it is necessary to make changes to a published set of source files or to try out a new idea. This type of build offers the most flexibility and allows users to experiment by adding their own code. In this type of build the DEM reads all its information from the file system, performs the build and sources any data load requirements from the file system. Every aspect of this type of build is driven from the file system and can easily be changed by the user. The build will however, only be able to operate against a single named environment for which the user, who is performing the build, has controlling rights.
Web Build
A build requested through the web-based interface would normally be used to build an environment that has to be built to a published specification. (e.g. a developer may need the latest schema build to work on a code change). This type of build takes the complete definition of an environment from source data which is held within the database, in the DEM's definition tables. Both the schema definition and the environment's data, are sourced from within the database. This is the most common type of build and is most useful in enhancing team productivity. For example, one person might be responsible for publishing the project's data model. Once built into the DEM, all developers may then request a re-build of their environment without asking for any assistance. Their working environments will then be dropped and re-built to the latest standard by the DEM. All Web Builds require a Repository from which to copy the structure and data.
Repository Build
A Repository Build is the way in which a build is 'published' within the DEM. It is a special type of build that would normally be performed by the DBA or a Data Modeller. When a new version of a project's database model is published (i.e. The source is assumed to be in a configuration managed repository somewhere), that code is extracted from the source control system and built as a Repository Build. The DEM performs a Custom Build for a repository and then performs some extra internal operations that allow this build to be copied by users wishing to perform a Web Build. Once built, a Repository may source any number of copy environments and Schemation keeps track of which environments are based on Repositories. With this implementation model, the DEM is able to free the DBA, or developer from all the mundane tasks involved in rebuilding environments. The DEM also provides an automatic system for refreshing Web Build environments when a Repository is updated.
Actions of the DEM during a Build
All database environment builds performed by the DEM will involve the following set of actions:
1. Clear down existing environment by dropping all associated database users and their objects.
2. Create all database user accounts defined for the environment. (Some environments require a group of users.)
3. Grant required system privileges and external object privileges to these accounts. (e.g. create table and execute on sys.dbms_flashback)
4. Create database tables, indexes, stored procedures and any other database objects defined for the environment.
5. Determine the environment's data load order (taking into account referential integrity constraints).
6. Load all requested data in the correct order (either from the operating system or from a data repository).
7. Instantiate the environment's security model by granting object privileges between the different users defined for the environment.
During the execution of the build the DEM records, in real time, its progress for each activity. This information is available for monitoring both during and after the build. Almost every operation that the DEM performs for any build is recorded in DEM's logging tables together with the result (pass/fail) and any associated error message. This is then centrally visible using the web-based interface. The Custom Build is one in which an entire database environment is created from the operating system. There are some basic terms which you must be familiar with in order to understand the process. These are discussed below.
Projects
Projects in the Schemation DEM are synonymous with projects in the business sense. They represent divisions of work into manageable chunks each of which contain deliverables. Projects are created in the DEM using the web-based administrative screens provided to administrators of the DEM. They provide a means to divide and identify environments. Certain reports that the DEM can produce may be made at the project level.
If your company situation is one where multiple projects are developed on shared equipment, then you should use DEM projects to separate database environments. If your project has the luxury of exclusive equipment, then you will only need to define a single DEM project and work within this project.
Within a project there arises the need for multiple database environments. These environments provide individual and complete database schemas or schema groups which offer the provisions of the project's database. An environment forms the complete database solution and is created for a specific purpose. Examples of environments in typical usage are: Individual code developer's environments; Data development environments; Administrative function development environments; and Test and Quality Assurance environments. The merits of duplicating database environments are clear, as they provide trusted resources where the activities of other developers and testers (working in their own environments) may be isolated from one and other.
With the DEM it is very simple to create and manage new environments, and with the complexities of creating many duplicate environments reduced, more project people may have their own environments and this leads to a better working environment for administrators, developers and testers.
On creation, every project within the DEM is given a unique, its 'Project-id'.
Environments
An environment is the name given to a database schema, or possibly several database schemas linked together, which hold tables, indexes, stored program units etc, and which provide a database resource for a project or application.
To create an environment, it is necessary to use the DEM administration console (web-based). Only administrative accounts of the DEM have the privilege to 'create new environments.
Once 'created' the environment is just a template with no real substance. A few properties will be defined for it, but at this point it will not be associated with any real database schemas - until it has been built.
The process of running or performing a build relates to one and only one environment at a time.
Environment Names
When a project is defined, it is given an abbreviation called a Project-id. An environment is also given an abbreviation - the Environment-id, but the environment's full identification includes the project's identifier too. The 'complete identifier for an environment is then made by combining both the Project-id and the Environment-id into a single unique identifier, called the PENV.
This becomes the unique identifier for an environment within Schemation's DEM. It's name is just a shorthand way of writing the Project-id and the Environment-id. All environments have this form in Schemation DEM. The two components are combined using a single underscore character ‘_’.
PENV = {project-id}_{environment-id)
Restrictions of Project-Ids, Environment-Ids and PENVs
* A project id must be unique within a DEM repository.
* A project can have many environments.
* An environment _id within a project must be unique. (i.e the combination of project id and environment id must be unique).
* Proiect ids and environment ids on there own are not used by Schemation, but they give rise to the PENV which is used to identify an environment.
Example: for the environment UAT1 within the project IDB, the PENV would be IDB_UAT1
The Importance of the PENV
The PENV uniquely identifies an environment in Schemation's DEM. As such, the PENV is used whenever you need to operate or produce reports on a specific environment. In the operating system, the PENV is referred to as SCH_PENV.
All Schemation environment parameters have the SCH_ prefix to differentiate them from other application parameters which may have the same name. As will be seen later, the Schemation DEM Custom Build process examines the SCH_PENV environment parameter when invoked from the operating system.