DSpace System Documentation: Configuration and Customization

Back to contents

Configuration and Customization Table of Contents

• General Configuration
• dspace.cfg
• Search Indexes
• Browse Configuration
• Configuring Media Filters
• Email Messages
• Metadata and Bitstream Format Registries
• Default Submission License
• Submission Configuration
  • Understanding the Submission Configuration File
  • Reordering/Removing Submission Steps
  • Assigning a custom Submission Process to a Collection
  • Customizing the Metadata-entry pages
  • Configuring the File Upload step
  • Creating new Submission Steps
• XMLUI Interface Customizations (Manakin)
  • dspace.cfg
  • Configuring Themes and Aspects
  • Multilingual Support
  • Creating a New Theme
• JSPUI Interface Customizations
• dspace.cfg
• Configuring Controlled Vocabularies
• Configuring Multilingual Support
• Customizing the JSP Pages
• Advanced DSpace Customizations
• Checksum Checker
• Custom Authentication
• Configuring Statistical Reports
• Activating Additional OAI-PMH Crosswalks
• Configuring Packager Plugins
• Configuring Crosswalk Plugins
• Creating new Media/Format Filters
• Configuration Files for Other Applications
• Browse Index Creation

There are a number of ways in which DSpace can be configured and/or customized:

• Altering the configuration files in [dspace]/config
• Creating a new XMLUI (Manakin) theme to change the look-and-feel of the repository
• Creating modified versions of the JSP pages for local changes in the JSPUI interface
• Implementing a custom 'plug-in' class -- for example, an 'authenticator' class, so that user authentication in the Web UI can be adapted and integrated with any existing mechanisms your organization might use, or a 'media filter' to generate thumbnails or extract full text from a new file format
• Editing the source code

Of these methods, only the last is likely to cause any headaches; if you update the DSpace source code directly, particularly core class files in org.dspace.* or org.dspace.storage.*, it may make applying future updates difficult. Before doing this, it is strongly recommended that you e-mail the DSpace developer community to find out the best way to proceed, and the best way to implement your change in a way that can be contributed back to DSpace for everyone's benefit.

General Configuration

These are general configuration options that apply to the core of DSpace regardless of which interface you are using (JSPUI or XMLUI).

The dspace.cfg Configuration Properties File

The primary way of configuring DSpace is to edit the dspace.cfg. You'll definitely have to do this before you can operate DSpace properly. dspace.cfg contains basic information about a DSpace installation, including system path information, network host information, and other things like site name.

The default dspace.cfg is a good source of information, and contains comments for all properties. It's a basic Java properties file, where lines are either comments, starting with a '#', blank lines, or property/value pairs of the form:

property.name = property value

The property value may contain references to other configuration properties, in the form ${property.name}. This follows the ant convention of allowing references in property files. A property may not refer to itself. Examples:

property.name = word1 ${other.property.name} more words
property2.name = ${dspace.dir}/rest/of/path

Property values can include other, previously defined values, by enclosing the property name in ${...}. For example, if your dspace.cfg contains: -

  dspace.dir = /dspace
  dspace.history = ${dspace.dir}/history
  

Then the value of the dspace.history property is expanded to be /dspace/history. This method is especially useful for handling commonly used file paths.

Whenever you edit dspace.cfg, you should then run [dspace]/bin/install-configs so that any changes you may have made are reflected in the configuration files of other applications, for example Apache. You may then need to restart those applications, depending on what you changed.

Configuring Lucene Search Indexes

Search Indexes can be configured via the dspace.cfg file. This allows institutions to choose which DSpace metadata fields are indexed by Lucene.

For example, the following entries appear in a default DSpace installation:

search.index.1 = author:dc.contributor.*
search.index.2 = author:dc.creator.*
search.index.3 = title:dc.title.*
search.index.4 = keyword:dc.subject.*
search.index.5 = abstract:dc.description.abstract
search.index.6 = author:dc.description.statementofresponsibility
search.index.7 = series:dc.relation.ispartofseries
search.index.8 = abstract:dc.description.tableofcontents
search.index.9 = mime:dc.format.mimetype
search.index.10 = sponsor:dc.description.sponsorship
search.index.11 = id:dc.identifier.*
	

The form of each entry is search.index.<id> = <search <schema>field>:<metadata field> where:

• <id> is an incremental number to distinguish each search index entry
• <search field> is an identifier for the search field this index will correspond to
• <metadata field> is the DSpace metadata field to be indexed

So in the example above, search.indexes1, 2 and 6 are configured as the author search field. The author index is created by Lucene indexing all contributor, creator and description.statementofresponsibility medatata fields.

After changing the configuration, run index-all to recreate the indexes.

NOTE: While the indexes are created, this only affects the search results and has no effect on the search components of the user interface. To add new search capability (e.g. to add a new search category to the Advanced Search) requires local customisation to the user interface.

Browse Configuration

The browse indices for DSpace can be extensively configured. This section of the configuration allows you to take control of the indices you wish to browse on, and how you wish to present the results. This configuration is broken down into several parts: defining the indices, defining the fields upon which users can sort results, defining truncation for potentially long fields (e.g. author lists), setting cross-links between different browse contexts (e.g. from an author's name to a complete list of their items), how many recent submissions to display, and configuration for item mapping browse.

Defining the Indices

The form is:

webui.browse.index.<n> = <index name> : \
                               <schema prefix>.<element>[.<qualifier>|.*] : \
                               (date | title | text) : \
                               (full | single) \

index name

The name by which the index will be identified. This may be used in later configuration or to locate the message key for this index.

<schema prefix>.<element>[.<qualifier>|.*]

The metadata field declaration for the field to be indexed. This will be something like dc.date.issued or dc.contributor.* or dc.title.

(date | title | text)

This refers to the datatype of the field:

• date: the index type will be treated as a date object
• title: the index type will be treated like a title, which will include a link to the item page
• text: the index type will be treated as plain text. If single mode is specified then this will link to the full mode list

(full | single)

This refers to the way that the index will be displayed in the browse listing. "Full" will be the full item list as specified by webui.itemlist.columns; "single" will be a single list of only the indexed term.

If you are customising this list beyond the default you will need to insert the text you wish to appear in the navigation and on link and buttons describing the browse index into the Messages.properties file. The system uses parameters of the form:

browse.type.<index name>

The Index numbers denoted by <n> must start from 1 and increment by 1 continuously thereafter. Deviation from this rule will cause an error during installation or during configuration update

This is an example configuration, as it appears by default in dspace.cfg.

webui.browse.index.1 = dateissued:dc.date.issued:date:full
webui.browse.index.2 = author:dc.contributor.*:text:single
webui.browse.index.3 = title:dc.title:title:full
webui.browse.index.4 = subject:dc.subject.*:text:single
webui.browse.index.5 = dateaccessioned:dc.date.accessioned:date:full

Defining Sort Options

Sort options will be available when browsing a list of items (i.e. only in "full" mode, not "single" mode). You can define an arbitrary number of fields to sort on, irrespective of which fields you display using webui.itemlist.columns

The format is:

webui.browse.sort-option.<n> = <option name> : \
                               <schema prefix>.<element>[.<qualifier>|.*] : \
                               (date | text)

option name

The name by which the sort option will be identified. This may be used in later configuration or to locate the message key for this index.

<schema prefix>.<element>[.<qualifier>|.*]

The metadata field declaration for the field to be sorted on. This will be something like dc.title or dc.date.issued.

(date | text)

This refers to the datatype of the field:

• date: the sort type will be treated as a date object
• text: the sort type will be treated as plain text.

This is the example configuration as it appears in the default dspace.cfg:

				
webui.browse.sort-option.1 = title:dc.title:text
webui.browse.sort-option.2 = date:dc.date.issued:date

Author (Multiple metadata value) Display

(Note: this section actually applies to any field with multiple values, but authors are the defined case)

You can define which field is the author (or editor, or other repeated field) which this configuration will deal with thus:

webui.browse.author-field = dc.contributor.*

The field should be listed in the configuration for webui.itemlist.columns, otherwise you will not see its effect. It must also be defined in webui.itemlist.columns as being of data type text, otherwise the functionality will be overriden by the specific data type features.

Now that we know which field is our author or other multiple metadata value field we can provide the option to truncate the number of values displayed by default. We replace the remaining list of values with "et al" or the language pack specific alternative. Note that this is just for the default, and users will have the option of changing the number displayed when they browse the results

webui.browse.author-limit = <n>

Where <n> is an integer number of values to be displayed. Use -1 for unlimited (default).

Links to other browse contexts

We can define which fields link to other browse listings. This is useful, for example, to link an author's name to a list of just that author's items. The effect this has is to create links to browse views for the item clicked on. If it is a "single" type, it will link to a view of all the items which share that metadata element in common (i.e. all the papers by a single author). If it is a "full" type, it will link to a view of the standard full browse page, starting with the value of the link clicked on.

The form is:

webui.browse.link.<n> = <index name>:<display column metadata>

This should associated the name of one of the browse indices (webui.browse.index.n) with a metadata field listed in webui.itemlist.columns above. If this condition is not fulfilled, cross-linking will not work. Note also that cross-linking only works for metadata fields not tagged as title in webui.itemlist.columns.

The following example shows the default in dspace.cfg which links author names to lists of their publications:

webui.browse.link.1 = author:dc.contributor.*

Recent Submissions

This allows us to define which index to base Recent Submission display on, and how many we should show at any one time. This uses the PluginManager to automatically load the relevant plugin for the Community and Collection home pages. Values given in examples are the defaults supplied in dspace.cfg

First define the sort name (from webui.browse.sort-option) to use for # displaying recent submissions. For example:

recent.submissions.sort-option = dateaccessioned

Define how many recent submissions should be displayed at any one time, for example:

recent.submissions.count = 5

Now we need to set up the processors that the PluginManager will load to actually perform the recent submissions query on the relevant pages.

Tell the community and collection pages that we are using the Recent Submissions code

plugin.sequence.org.dspace.plugin.CommunityHomeProcessor = \
	org.dspace.app.webui.components.RecentCommunitySubmissions

plugin.sequence.org.dspace.plugin.CollectionHomeProcessor = \
	org.dspace.app.webui.components.RecentCollectionSubmissions

This is already configured by default in dspace.cfg so there should be no need for you to worry about it

Item Mapper

Because the item mapper requires a primitive implementation of the browse system to be present, we simply need to tell that system which of our indices defines the author browse (or equivalent) so that the mapper can list authors' items for mapping

Define the the index name (from webui.browse.index) to use for displaying items by author

itemmap.author.index = author

So if you change the name of your author browse field, you will also need to update this configuration.

Configuring Media Filters

Media or Format Filters are classes used to generate derivative or alternative versions of content or bitstreams within DSpace. For example, the PDF Media Filter will extract textual content from PDF bitstreams, the JPEG Media Filter can create thumbnails from image bitstreams.

Media Filters are configured as Named Plugins, with each filter also having a separate configuration setting (in dspace.cfg) indicating which formats it can process. The default configuration is shown below.

	#### Media Filter / Format Filter plugins (through PluginManager) ####

	#Names of the enabled MediaFilter or FormatFilter plugins
	filter.plugins = PDF Text Extractor, HTML Text Extractor, \
						Word Text Extractor, JPEG Thumbnail
	# to enable branded preview: remove last line above, and uncomment 2 lines below
	#   					Word Text Extractor, JPEG Thumbnail, \
	#   					Branded Preview JPEG

	#Assign 'human-understandable' names to each filter
	plugin.named.org.dspace.app.mediafilter.FormatFilter = \
  		org.dspace.app.mediafilter.PDFFilter = PDF Text Extractor, \
  		org.dspace.app.mediafilter.HTMLFilter = HTML Text Extractor, \
  		org.dspace.app.mediafilter.WordFilter = Word Text Extractor, \
  		org.dspace.app.mediafilter.JPEGFilter = JPEG Thumbnail, \
  		org.dspace.app.mediafilter.BrandedPreviewJPEGFilter = Branded Preview JPEG

	#Configure each filter's input format(s)
	filter.org.dspace.app.mediafilter.PDFFilter.inputFormats = Adobe PDF
	filter.org.dspace.app.mediafilter.HTMLFilter.inputFormats = HTML, Text
	filter.org.dspace.app.mediafilter.WordFilter.inputFormats = Microsoft Word
	filter.org.dspace.app.mediafilter.JPEGFilter.inputFormats = GIF, JPEG, image/png
	filter.org.dspace.app.mediafilter.BrandedPreviewJPEGFilter.inputFormats = GIF, JPEG, image/png

The enabled Media/Format Filters are named in the filter.plugins field above.

Names are assigned to each filter using the plugin.named.org.dspace.app.mediafilter.FormatFilter field (e.g. by default the PDFFilter is named "PDF Text Extractor").

Finally the appropriate filter.<class path>.inputFormats defines the vaild input formats which each filter can be applied to. These format names must match the short description field of the Bitstream Format Registry.

You can also implement more dynamic or configurable Media/Format Filters which extend SelfNamedPlugin. More information is provide below in Creating a new Media/Format Filter

Wording of E-mail Messages

Sometimes DSpace automatically sends e-mail messages to users, for example to inform them of a new workflow task, or as a subscription e-mail alert. The wording of emails can be changed by editing the relevant file in [dspace]/config/emails. Each file is commented. Be careful to keep the right number 'placeholders' (e.g.{2}).
Note: You should replace the contact-information "dspace-help@myu.edu or call us at xxx-555-xxxx" with your own contact details in:

• config/emails/change_password
• config/emails/register

The Metadata and Bitstream Format Registries

The [dspace]/config/registries directory contains three XML files. These are used to load the initial contents of the Metadata Schema Registry, Dublin Core Metadata registry and Bitstream Format registry. After the initial loading (performed by ant fresh_install above), the registries reside in the database; the XML files are not updated.

In order to change the registries, you may adjust the XML files before the first installation of DSpace. On an allready running instance it is recommended to change bitstream registries via DSpace admin UI, but the metadata registries can be loaded again at any time from the XML files without difficult. The changes made via admin UI are not reflected in the XML files.

Metadata Schema Registry

The default metadata schema in DSpace is Dublin Core, so it is distributed with a single entry in the source XML file for that namespace. If you wish to add more schemas you can do this in one of two ways. Via the DSpace admin UI you may define new Metadata Schemas, edit existing schemas and move elements between schemas. But you may also modify the XML file (or provide an additional one), and re-import the data as follows:

  [dspace]/bin/dsrun org.dspace.adminster.SchemaImporter -f [xml file]
  

	<metadata-schemas>
		<schema>
			<name>[schema name]</name>
			<namespace>http://myu.edu/some/namespace</namespace>
		</schema>
	</metadata-schemas>
  

Metadata Format Registries

The default metadata schema is Dublin Core, so DSpace is distributed with a default Dublin Core Metadata Registry. Currently, the system requires that every item have a Dublin Core record.

There is a set of Dublin Core Elements, which is used by the system and should not be removed or moved to another schema, see Appendix: Default Dublin Core Metadata registry.

Note: altering a Metadata Registry has no effect on corresponding parts, e.g. item submission interface, item display, item import and vice versa. Every metadata element used in submission interface or item import must be registered before using it.

Note also that deleting a metadata element will delete all its corresponding values.

If you wish to add more metadata elements, you can do this in one of two ways. Via the DSpace admin UI you may define new metadata elements in the different available schemas. But you may also modify the XML file (or provide an additional one), and re-import the data as follows:

  [dspace]/bin/dsrun org.dspace.adminster.MetadataImporter -f [xml file]
  

	<dspace-dc-types>
		<dc-type>
			<schema>dc</schema>
			<element>contributor</element>
			<qualifier>advisor</qualifier>
			<scope_note>Use primarily for thesis advisor.</scope_note>
		</dc-type>
	</dspace-dc-types>
 	

Bitstream Format Registry

The bitstream formats recognized by the system and levels of support are similarly stored in the bitstream format registry. This can also be edited at install-time via [dspace]/config/registries/bitstream-formats.xml or by the administation Web UI. The contents of the bitstream format registry are entirely up to you, though the system requires that the following two formats are present:

• Unknown
• License

The Default Submission License

For each submitted item, a license must be granted. The license will be stored along with the item in the bundle LICENSE in order to keep the information under which terms an items has been published.

You may define a license for each collection seperately, when creating/editing a collection. If no collection specific license is defined, the default license is used.

The default license can be found in [dspace]/config/default.license and can be edited via the dspace-admin interface.

DSpace comes with a demo license, which you must adopt to your institutional needs and the legal regulations of your country.

If in doubt, contact the law department of your institution.

Possible Points in a License

• Non-exclusive or exclusive right to
• capture and store
• distribute
• worldwide
• restricted (e.g. institutional wide
• translate
• transform to other formats or mediums
without changing the content
• Make sure no rights (copyright or any other) are violated by this publication
• In case the type of submission (e.g. thesis) needs approval, make sure it is the final and approved version.
• Distinguish between the document itself and the metadata
• Point out that the license granted and the information who granted it will be stored.

Submission Configuration

Instructions for customizing and configuring the Item Submission user interface for either the JSP-UI or XML-UI are contained in the separate Customizing and Configuring Submission User Interface page.

XMLUI Interface Customizations (Manakin)

XMLUI Configuration Properties

There are several options effecting how the XMLUI user interface for DSpace operates. Listed below are the major elements and their description, refere to the dspace.cfg file itself for the exhaustive list of configuration parameters.

Configuring Themes and Aspects

   <aspects>
     <aspect name="Artifact Browser" path="resource://aspects/ArtifactBrowser/" />
     <aspect name="Administration" path="resource://aspects/Administrative/" />
     <aspect name="E-Person" path="resource://aspects/EPerson/" />
     <aspect name="Submission and Workflow" path="resource://aspects/Submission/" />
   </aspects>

  <themes>
    <theme name="Theme 1" handle="123456789/23" path="theme1/"/>
    <theme name="Theme 2" regex="community-list" path="theme2/"/>
    <theme name="Reference Theme" regex=".*" path="Reference/"/>
  </themes>

Multilingual Support

The XMLUI user interface supports multiple languages through the use of internationalization catalogues as defined by the Cocoon Internationalization Transformer. Each catalogue contains the translation of all user-displayed strings into a particular language or variant. Each catalogue is a single xml file whose name is based upon the language it is designated for, thus:

The interface will automatically determine which file to select based upon the user's browser and system configuration. For example, if the user's browser is set to Australian English then first the system will check if messages_en_au.xml is available. If this translation is not available it will fall back to messages_en.xml, and finally if that is not available, messages.xml.

Manakin supplies an English only translation of the interface. In order to add other translations to the system, locate the [dspace-source]/dspace/modules/xmlui/src/main/webapp/i18n/ directory. By default this directory will be empty; to add additional translations add alternative versions of the messages.xml file in specific language and country variants as needed for your installation.

Creating a New Theme

Manakin themes stylize the look-and-feel of the repository, community, or collection and are distributed as self-contained packages. A Manakin/DSpace installation may have multiple themes installed and available to be used in different parts of the repository. The central component of a theme is the sitemap.xmap, which defines what resources are available to the theme such as XSL stylesheets, CSS stylesheets, images, or multimedia files.

Most theme developers do not create a new theme from scratch; instead they start from the standard theme template, which defines a skeleton structure for a theme. The template is located at: [dspace-source]/dspace-xmlui/dspace-xmlui-webbapp/src/main/webbapp/themes/template. To start your new theme simply copy the theme template into your locally defined modules directory, [dspace-source]/dspace/modules/xmlui/src/main/webbapp/themes/[your theme's directory]/.

The next step is to modify the theme's parameters so that the theme knows where it is located. Open the [your theme's directory]/sitemap.xmap and look for <global-variables>

   <global-variables>
      <theme-path>[your theme's directory]</theme-path>
      <theme-name>[your theme's name]</theme-name>
   </global-variables>

Update both the theme's path to the directory name you created in step one. The theme's name is used only for documentation.

The base theme template will produce a repository interface without any style - just plain XHTML with no color or formatting. To make your theme useful you will need to supply a CSS Stylesheet that creates your desired look-and-feel. Add your new CSS stylesheets:

[your theme's directory]/lib/style.css (The base style sheet used for all browsers)
[your theme's directory]/lib/style-ie.css (Specific stylesheet used for internet explorer)

4) Install theme and rebuild DSpace

Next rebuild & deploy DSpace as described in the installation portion of the manual, and ensure the theme has been installed as described in the previous section "Configuring Themes and Aspects".

JSPUI Interface Customizations

DSpace digital repository supports two user interfaces one based upon JSP technologies and the other based upon the Apache Cocoon framework. This section describes those parameters which are specific to the JSPUI interface.

JSPUI Configuration Properties

There are many options effecting how the JSP-based user interface for DSpace operates. Listed below are the major elements and their description, refere to the dspace.cfg file itself for the exhaustive list of configuration parameters.

Configuring Controlled Vocabularies

DSpace now supports controlled vocabularies to confine the set of keywords that users can use while describing items.

The need for a limited set of keywords is important since it eliminates the ambiguity of a free description system, consequently simplifying the task of finding specific items of information.

The controlled vocabulary add-on allows the user to choose from a defined set of keywords organised in an tree (taxonomy) and then use these keywords to describe items while they are being submitted.

We have also developed a small search engine that displays the classification tree (or taxonomy) allowing the user to select the branches that best describe the information that he/she seeks.

The taxonomies are described in XML following this (very simple) structure:

<node id="acmccs98" label="ACMCCS98">
  <isComposedBy>
   <node id="A." label="General Literature">
    <isComposedBy>
     <node id="A.0" label="GENERAL"/>
     <node id="A.1" label="INTRODUCTORY AND SURVEY"/>
     ...
    </isComposedBy>
   </node>
   ...
  </isComposedBy>
</node>


Your are free to use any application you want to create your controlled vocabularies. A simple text editor should be enough for small projects. Bigger projects will require more complex tools. You may use Protegé to create your taxonomies, save them as OWL and then use a XML Stylesheet (XSLT) to transform your documents to the appropriate format. Future enhancements to this add-on should make it compatible with standard schemas such as OWL or RDF.

In order to make DSpace compatible with WAI 2.0, the add-on is turned off by default (the add-on relies strongly on Javascript to function). It can be activated by setting the following property in dspace.cfg:

webui.controlledvocabulary.enable = true

New vocabularies should be placed in [dspace]/config/controlled-vocabularies/ and must be according to the structure described. A validation XML Schema can be downloaded here.

Vocabularies need to be associated with the correspondant DC metadata fields. Edit the file [dspace]/config/input-forms.xml and place a "vocabulary" tag under the "field" element that you want to control. Set value of the "vocabulary" element to the name of the file that contains the vocabulary, leaving out the extension (the add-on will only load files with extension "*.xml"). For example:

<field>
    <dc-schema>dc</dc-schema>
    <dc-element>subject</dc-element>
    <dc-qualifier></dc-qualifier>
    <!-- An input-type of twobox MUST be marked as repeatable -->
    <repeatable>true</repeatable>
    <label>Subject Keywords</label>
    <input-type>twobox</input-type>
    <hint> Enter appropriate subject keywords or phrases below. </hint>
  <required></required>
  <vocabulary [closed="false"]>nsi</vocabulary>
</field>


The vocabulary element has an optional boolean attribute closed that can be used to force input only with the javascript of controlled-vocabulary add-on. The default behaviour (i.e. without this attribute) is as set closed="false". This allow the user also to enter the value in free way.

The following vocabularies are currently available by default:

• nsi - nsi.xml - The Norwegian Science Index
• srsc - srsc.xml - Swedish Research Subject Categories

Configuring Multilingual Support

Setting the default language for the application

Supporting more than one language

Changes in dspace.cfg

• a language switch in the default header
• the user will able to choose his preferred language, this will be part of his profile
• wording of emails
  • mails to registered users e. g. alerting service will use the preferred language of the user
  • mails to unregistered users e. g. suggest an item will use the language of the session
• according to the language selected for the session, using dspace-admin Edit News will edit the news file of the language according to session

Related files

• [dspace]/modules/jspui/src/main/resources/Messages.properties
• [dspace]/modules/jspui/src/main/resources/Messages_en.properties
• [dspace]/modules/jspui/src/main/resources/Messages_de.properties
• [dspace]/modules/jspui/src/main/resources/Messages_fr.properties

• [dspace]/modules/jspui/src/main/resources/Messages_LOCALE.properties
• [dspace]/config/input-forms_LOCALE.xml
• [dspace]/config/default_LOCALE.license should be pure ascii
• [dspace]/config/news-top_LOCALE.html
• [dspace]/config/news-side_LOCALE.html
• [dspace]/config/emails/change_password_LOCALE
• [dspace]/config/emails/feedback_LOCALE
• [dspace]/config/emails/internal_error_LOCALE
• [dspace]/config/emails/register_LOCALE
• [dspace]/config/emails/submit_archive_LOCALE
• [dspace]/config/emails/submit_reject_LOCALE
• [dspace]/config/emails/submit_task_LOCALE
• [dspace]/config/emails/subscription_LOCALE
• [dspace]/config/emails/suggest_LOCALE
• [dspace]/jsp/help/collection-admin_LOCALE.html in html keep the jump link as original
• [dspace]/jsp/help/index_LOCALE.html
• [dspace]/jsp/help/site-admin_LOCALE.html

Customizing the JSP pages

The JSPUI interface is implemented using Java Servlets which handle the business logic, and JavaServer Pages (JSPs) which produce the HTML pages sent to an end-user. Since the JSPs are much closer to HTML than Java code, altering the look and feel of DSpace is relatively easy.

To make it even easier, DSpace allows you to 'override' the JSPs included in the source distribution with modified versions, that are stored in a separate place, so when it comes to updating your site with a new DSpace release, your modified versions will not be overwritten. It should be possible to dramatically change the look of DSpace to suit your organization by just changing the CSS style file and the site 'skin' or 'layout' JSPs in jsp/layout; if possible, it is recommended you limit local customizations to these files to make future upgrades easier.

You can also easily edit the text that appears on each JSP page by editing the dictionary file. However, note that unless you change the entry in all of the different language message files, users of other languages will still see the default text for their language. See internationalization.

Note that the data (attributes) passed from an underlying Servlet to the JSP may change between versions, so you may have to modify your customized JSP to deal with the new data.

Thus, if possible, it is recommeded you limit your changes to the 'layout' JSPs and the stylesheet.

The JSPs are available in one of two places:

• [dspace-source]/dspace-jspui/dspace-jspui-webapp/src/main/webapp/ - Only exists if you downloaded the full Source Release of DSpace
• [dspace-source]/dspace/target/dspace-[version].dir/webapps/dspace-jspui-webapp/ - The location where they are copied after first building DSpace.

If you wish to modify a particular JSP, place your edited version in the [dspace-source]/dspace/modules/jspui/src/main/webapp/ directory (this is the replacement for the pre-1.5 /jsp/local directory), with the same path as the original. If they exist, these will be used in preference to the default JSPs. For example:

Heavy use is made of a style sheet, styles.css.jsp. If you make edits, copy the local version to [dspace-source]/dspace/modules/jspui/src/main/webapp/styles.css.jsp, and it will be used automatically in preference to the default, as described above.

Fonts and colors can be easily changed using the stylesheet. The stylesheet is a JSP so that the user's browser version can be detected and the stylesheet tweaked accordingly.

The 'layout' of each page, that is, the top and bottom banners and the navigation bar, are determined by the JSPs /layout/header-*.jsp and /layout/footer-*.jsp. You can provide modified versions of these (in [dspace-source]/dspace/modules/jspui/src/main/webapp/layout), or define more styles and apply them to pages by using the "style" attribute of the dspace:layout tag.

After you've customized your JSPs, you must rebuild the DSpace Web application. If you haven't already built and installed it, follow the install directions. Otherwise, follow the steps below:

1. Rebuild the DSpace installation package by running the following command from your [dspace-source]/dspace/ directory:

   mvn package

2. Re-install the DSpace WAR(s) to [dspace]/webapps by running the following command from your [dspace-source]/dspace/target/dspace-[version].dir directory:

   ant -Dconfig=[dspace]/config/dspace.cfg update
   

3. Depending on your setup with Tomcat, you may also need to do the following:

   • Shut down Tomcat, and delete any existing [tomcat]/webapps/dspace directories.
   • Copy the new .war file(s) to the Tomcat webapps directory:
   • Restart Tomcat.

When you restart the web server you should see your customized JSPs.

Advanced DSpace Customizations

Some customizations to the DSpace platform require advanced skills or knowlege to complete. The options list here will require either knowlege in system administration or may involve light programming.

Checksum Checker

There are three aspects of the Checksum Checker's operation that can be configured:

1. the execution mode
2. the logging output
3. the policy for removing old checksum results from the database

Checker Execution Mode

Execution mode can be configured using command line options. Information on the options can be found at any time by running [dspace]/bin/checker --help. The different modes are described below; see the "Which to use" section that follows for details on the various pros and cons.

Unless a particular bitstream or handle is specified, the Checksum Checker will always check bitstreams in order of the least recently checked bitstream. (Note that this means that the most recently ingested bitstreams will be the last ones checked by the Checksum Checker.)

To check a specific number of bitstreams, use the -c option followed by an integer number of bitstreams to check:

bin/checker -c 10

Limited count mode is particularly useful for checking that the checker is executing properly. The Checksum Checker's default execution mode is to check a single bitstream, as if the -c 1 option had been given.

To run the Checker for a specific period of time, use the -d option with a time argument:

bin/checker -d 10m
bin/checker -d 2h

Valid options for specifying duration are s for seconds, m for minutes, h for hours, d for days, w for weeks, and y for years (OK, so we're optimists).

The checker will keep starting new bitstream checks for the specified duration, so actual execution duration will be slightly longer than the specified duration. Bear this in mind when scheduling checks.

To check one or more particular bitstreams by ID, use the -b option followed by one or more bitstream IDs:

bin/checker -b 1 2 3 4

This mode is useful when analyzing problems reported in the logs and when verifying that a resolution has been successful.

Use the -a option followed by a handle:

bin/checker -a 123456/123

This will check all the bitstreams inside an item, collection or community.

There are two looping modes:

bin/checker -l     # Loops once through the repository  
bin/checker -L	    # Loops continuously through the repository

The -l option can be used if your repository is relatively small and your backup strategy requires it to be completely validated at a particular point. The -L option might be useful if you have a large repository, and you don't mind (or can avoid) the IO load caused by the checker.

The Checksum Checker was designed with the idea that most sys admins will run it from the cron. For small repositories we recommend using the -l option in the cron. For larger repositories that cannot be completely checked in a couple of hours, we recommend the -d option in the cron.

Checker Reporting

Checksum Checker uses log4j to report its results. By default it will report to a log called [dspace]/log/checker.log, and it will report only on bitstreams for which the newly calculated checksum does not match the stored checksum. To report on all bitstreams checked regardless of outcome, use the -v (verbose) command line option:

bin/checker -l -v    #Loop through the repository once and
  report in detail about every bitstream checked.

To change the location of the log, or to modify the prefix used on each line of output, edit the [dspace]/config/templates/log4j.properties file and run [dspace]/bin/install_configs.

Checker Results Pruning

The Checksum Checker will store the result of every check in the checksum_history table. By default, successful checksum matches that are eight weeks old or older will be deleted when the -p command line option is used (unsuccessful ones will be retained indefinitely). The amount of time for which results are retained in the checksum_history table can be modified by one of two methods:

1. editing the retention policies in [dspace]/config/dspace.cfg OR
2. passing in a properties file containing retention policies when using the -p option.

Pruning is controlled by a number of properties, each of which describes a checksum result code, and the length of time for which results with that code should be retained. The format is checker.retention.[RESULT CODE]=[duration]. For example: -

checker.retention.CHECKSUM_MATCH=8w

indicates that successful checksum matches will be retained for eight weeks. Supported units of time are

(Note that these units are also used for describing durations for the -d limited duration mode.)

There is a special property, checker.retention.default, that is used to assign a default retention period.

To execute the pruning you must use the -p command line option (with or without a properties file). Checksum Checker will prune the history table before beginning new checks. We recommend that you use this option regularly, as the checksum_history table can grow very large without it.

Custom Authentication

Since many institutions and organizations have exisiting authentication systems, DSpace has been designed to allow these to be easily integrated into an existing authentication infrastructure. It keeps a series, or "stack", of authentication methods, so each one can be tried in turn. This makes it easy to add new authentication methods or rearrange the order without changing any existing code. You can also share authentication code with other sites.

The configuration property plugin.sequence.org.dspace.authenticate.AuthenticationMethod defines the authentication stack. It is a comma-separated list of class names. Each of these classes implements a different authentication method, or way of determining the identity of the user. They are invoked in the order specified until one succeeds.

An authentication method is a class that implements the interface org.dspace.authenticate.AuthenticationMethod. It authenticates a user by evaluating the credentials (e.g. username and password) he or she presents and checking that they are valid.

The basic authentication procedure in the DSpace Web UI is this:

1. A request is received from an end-user's browser that, if fulfilled, would lead to an action requiring authorization taking place.
2. If the end-user is already authenticated:
   • If the end-user is allowed to perform the action, the action proceeds
   • If the end-user is NOT allowed to perform the action, an authorization error is displayed.
   • If the end-user is NOT authenticated, i.e. is accessing DSpace anonymously:
3. The parameters etc. of the request are stored
4. The Web UI's startAuthentication method is invoked.
5. First it tries all the authentication methods which do implicit authentication (i.e. they work with just the information already in the Web request, such as an X.509 client certificate). If one of these succeeds, it proceeds from Step 2 above.
6. If none of the implicit methods succeed, the UI responds by putting up a "login" page to collect credentials for one of the explicit authentication methods in the stack. The servlet processing that page then gives the proffered credentials to each authentication method in turn until one succeeds, at which point it retries the original operation from Step 2 above.

Please see the source files AuthenticationManager.java and AuthenticationMethod.java for more details about this mechanism.

Authentication by Password

The default method org.dspace.authenticate.PasswordAuthentication has the following properties:

• Use of inbuilt e-mail address/password-based log-in. This is achieved by forwarding a request that is attempting an action requiring authorization to the password log-in servlet, /password-login. The password log-in servlet (org.dspace.app.webui.servlet.PasswordServlet contains code that will resume the original request if authentication is successful, as per step 3. described above.

• Users can register themselves (i.e. add themselves as e-people without needing approval from the administrators), and can set their own passwords when they do this

• Users are not members of any special (dynamic) e-person groups

• You can restrict the domains from which new users are able to regiser. To enable this feature, uncomment the following line from dspace.cfg: authentication.password.domain.valid = example.com Example options might be '@example.com' to restrict registration to users with addresses ending in @example.com, or '@example.com, .ac.uk' to restrict registration to users with addresses ending in @example.com or with addresses in the .ac.uk domain.

X.509 Certificate Authentication

The X.509 authentication method uses an X.509 certificate sent by the client to establish his/her identity. It requires the client to have a personal Web certificate installed on their browser (or other client software) which is issued by a Certifying Authority (CA) recognized by the web server.

1. See the HTTPS installation instructions to configure your Web server. If you are using HTTPS with Tomcat, note that the <Connector> tag must include the attribute clientAuth="true" so the server requests a personal Web certificate from the client.

2. Add the org.dspace.authenticate.X509Authentication plugin first to the list of stackable authentication methods in the value of the configuration key plugin.sequence.org.dspace.authenticate.AuthenticationMethod E.g.:

       plugin.sequence.org.dspace.authenticate.AuthenticationMethod = \
           org.dspace.authenticate.X509Authentication, \
           org.dspace.authenticate.PasswordAuthentication
     

3. You must also configure DSpace with the same CA certificates as the web server, so it can accept and interpret the clients' certificates. It can share the same keystore file as the web server, or a separate one, or a CA certificate in a file by itself. Configure it by one of these methods, either the Java keystore

     authentication.x509.keystore.path = path to Java keystore file
     authentication.x509.keystore.password = password to access the keystore
    

   ...or the separate CA certificate file (in PEM or DER format):

     authentication.x509.ca.cert = path to certificate file for CA whose client certs to accept.
    

4. Choose whether to enable auto-registration: If you want users who authenticate successfully to be automatically registered as new E-Persons if they are not already, set the authentication.x509.autoregister configuration property to true. This lets you automatically accept all users with valid personal certificates. The default is false.

Example of a Custom Authentication Method

Also included in the source is an implementation of an authentication method used at MIT, edu.mit.dspace.MITSpecialGroup. This does not actually authenticate a user, it only adds the current user to a special (dynamic) group called 'MIT Users' (which must be present in the system!). This allows us to create authorization policies for MIT users without having to manually maintain membership of the MIT users group.

By keeping this code in a separate method, we can customize the authentication process for MIT by simply adding it to the stack in the DSpace configuration. None of the code has to be touched.

You can create your own custom authentication method and add it to the stack. Use the most similar existing method as a model, e.g. org.dspace.authenticate.PasswordAuthentication for an "explicit" method (with credentials entered interactively) or org.dspace.authenticate.X509Authentication for an implicit method.

Configuring IP Authentication

You can enable IP authentication by adding its method to the stack in the DSpace configuration, e.g.:

plugin.sequence.org.dspace.authenticate.AuthenticationMethod = org.dspace.authenticate.IPAuthentication

You are than able to map DSpace groups to IP's in dspace.cfg by setting authentication.ip.GROUPNAME = iprange[, iprange ...], e.g:

  authentication.ip.MY_UNIVERSITY = 10.1.2.3, \            # Full IP
                                    13.5, \                # Partial IP
                                    11.3.4.5/24, \         # with CIDR
                                    12.7.8.9/255.255.128.0 # with netmask
  

Note: if the Groupname contains blanks you must escape the, e.g. Department\ of\ Statistics

Configuring LDAP Authentication

You can enable LDAP authentication by adding its method to the stack in the DSpace configuration, e.g.

plugin.sequence.org.dspace.authenticate.AuthenticationMethod = org.dspace.authenticate.LDAPAuthentication

If LDAP is enabled in the dspace.cfg file, then new users will be able to register by entering their username and password without being sent the registration token. If users do not have a username and password, then they can still register and login with just their email address the same way they do now.

If you want to give any special privileges to LDAP users, create a stackable authentication method to automatically put people who have a netid into a special group. You might also want to give certain email addresses special privileges. Refer to the Custom Authentication Code section above for more information about how to do this.

Here is an explanation of what each of the different configuration parameters are for:

• ldap.enable
  This setting will enable or disable LDAP authentication in DSpace. With the setting off, users will be required to register and login with their email address. With this setting on, users will be able to login and register with their LDAP user ids and passwords.
• webui.ldap.autoregister
  This will turn LDAP autoregistration on or off. With this on, a new EPerson object will be created for any user who successfully authenticates against the LDAP server when they first login. With this setting off, the user must first register to get an EPerson object by entering their ldap username and password and filling out the forms.
• ldap.provider_url = ldap://ldap.myu.edu/o=myu.edu
  This is the url to your institution's ldap server. You may or may not need the /o=myu.edu part at the end. Your server may also require the ldaps:// protocol.
• ldap.id_field = uid
  This is the unique identifier field in the LDAP directory where the username is stored.
• ldap.object_context = ou=people,o=myu.edu
  This is the object context used when authenticating the user. It is appended to the ldap.id_field and username. For example uid=username,ou=people,o=myu.edu. You will need to modify this to match your ldap configuration.
• ldap.search_context = ou=people
  This is the search context used when looking up a user's ldap object to retrieve their data for autoregistering. With ldap.autoregister turned on, when a user authenticates without an EPerson object we search the ldap directory to get their name and email address so that we can create one for them. So after we have authenticated against uid=username,ou=people,o=byu.edu we now search in ou=people for filtering on [uid=username]. Often the ldap.search_context is the same as the ldap.object_context parameter. But again this depends on your ldap server configuration.
• ldap.email_field = mail
  This is the ldap object field where the user's email address is stored. "mail" is the default and the most common for ldap servers. If the mail field is not found the username will be used as the email address when creating the eperson object.
• ldap.surname_field = sn
  This is the ldap object field where the user's last name is stored. "sn" is the default and is the most common for ldap servers. If the field is not found the field will be left blank in the new eperson object.
• ldap.givenname_field = givenName
  This is the ldap object field where the user's given names are stored. I'm not sure how common the givenName field is in different ldap instances. If the field is not found the field will be left blank in the new eperson object.
• ldap.phone_field = telephoneNumber
  This is the field where the user's phone number is stored in the ldap directory. If the field is not found the field will be left blank in the new eperson object.

Configuring System Statistical Reports

Currently the statistic's engine is only available for the JSP-based user interface

Statistics for the system can be made available at http://www.mydspaceinstance.edu/statistics. To use the system statistics you will have to initialise them as per the installation documentation, but before you do so you need to perform the customisations discussed here in order to ensure that the reports are generated correctly.

Configuration File

Configuration for the statistics system are in [dspace]/config/dstat.cfg and the file should guide you to correctly filling in the details required. For the most part you will not need to change this file.

Customising Shell Scripts

To customise the supplied perl scripts to do monthly and general report generation it is necessary to modify the scripts themselves sightly. This is because these scripts were developed to speed up the process of using DStat at Edinburgh University Library and were not particularly intended for external use. They appear here for the convenience of others and in order to bridge the gap between the report generation and the inclusion of those reports into the DSpace UI, which is currently a clunky process.

In order to get these scripts to work for you, open each of the following in turn:

stat-general
stat-initial
stat-monthly
stat-report-general
stat-report-initial
stat-report-monthly

scripts eding with -general do the work for building reports spanning the entire history of the archive; scripts ending -initial are to initialise the reports by doing monthly reports from some start date up to the present; scripts ending -monthly generate a single monthly report for the current month. These scripts are just designed to make life easier, and are not particularly clever or elegant.

In each file you will find a section:

# Details used
################################################

... some perl ...

################################################

the perl between the lines of hashes defines the variables which will be used to do all of the processing in the report. The following explains what the variables mean and what they should be set to for each of the scripts

stat-initial:
$out_prefix: prefix to place in front of each output file.
$out_suffix: suffix for output file. A date will be inserted between the prefix and suffix
$start_year: year to start back-analysing monthly logs from
$start_month: month to start back-analysing monthly logs from
$dsrun: path to your dsrun script, usually [dspace]/bin/dsrun
$out_directory: directory into which to place analysis files, for example [dspace]/bin/log/

stat-monthly:
$out_prefix: prefix to place in front of each output file.
$out_suffix: suffix for output file. A date will be inserted between the prefix and suffix
$dsrun: path to your dsrun script, usually [dspace]/bin/dsrun
$out_directory: directory into which to place analysis files, for example [dspace]/bin/log/

stat-general:
$out_prefix: prefix to place in front of each output file.
$out_suffix: suffix for output file. Today's date will be inserted between the prefix and suffix
$dsrun: path to your dsrun script, usually [dspace]/bin/dsrun
$out_directory: directory into which to place analysis files, for example [dspace]/bin/log/

stat-report-initial:
$in_prefix: the prefix of the files generated by stat-initial
$in_suffix: the suffix of the files generated by stat-initial
$out_prefix: the report file prefix. Should be "report-" in order to work with DSpace UI
$out_suffix: the report file suffix. Should be ".html" in order to work with DSpace UI
$start_year: the start year used in stat-initial
$start_month: the start month used in stat-initial
$dsrun: path to your dsrun script, usually [dspace]/bin/dsrun
$in_directory: directory where analysis files were placed in stat-initial
$out_directory: the live reports directory: [dspace]/reports/

stat-report-monthly:
$in_prefix: the prefix of the files generated by stat-monthly
$in_suffix: the suffix of the files generated by stat-monthly
$out_prefix: the report file prefix. Should be "report-" in order to work with DSpace UI
$out_suffix: the report file suffix. Should be ".html" in order to work with DSpace UI
$dsrun: path to your dsrun script, usually [dspace]/bin/dsrun
$in_directory: directory where analysis files were placed in stat-monthly
$out_directory: the live reports directory: [dspace]/reports/

stat-report-general:
$in_prefix: the prefix of the files generated by stat-general
$in_suffix: the suffix of the files generated by stat-general
$out_prefix: the report file prefix. Should be "report-general-" in order to work with DSpace UI
$out_suffix: the report file suffix. Should be ".html" in order to work with DSpace UI
$dsrun: path to your dsrun script, usually [dspace]/bin/dsrun
$in_directory: directory where analysis files were placed in stat-general
$out_directory: the live reports directory: [dspace]/reports/

If you want additional customisations, you will need to modify the lines which build the command to be executed and change the parameters passed to the java processes which actually carry out the analysis. For more information on these processes either build the javadocs or run:

[dspace]/bin/dsrun org.dspace.app.statistics.LogAnalyser -help
[dspace]/bin/dsrun org.dspace.app.statistics.ReportGenerator -help

Activating Additional OAI-PMH Crosswalks

DSpace comes with an unqualified DC Crosswalk used in the default OAI-PMH data provider. There are also other Crosswalks bundled with the DSpace distribution which can be activated by editing one or more configuration files. How to do this for each available Crosswalk is described below. The DSpace source includes the following crosswalk plugins available for use with OAI-PMH:

• mets - The manifest document from a DSpace METS SIP.
• mods - MODS metadata, produced by the table-driven MODS dissemination crosswalk.
• qdc - Qualfied Dublin Core, produced by the configurable QDC crosswalk. Note that this QDC does not include all of the DSpace "dublin core" metadata fields, since the XML standard for QDC is defined for a different set of elements and qualifiers.

OAI-PMH crosswalks based on Crosswalk Plugins are activated as follows:

1. Ensure the crosswalk plugin has a lower-case name (possibly in addition to its upper-case name) in the plugin configuration.
2. Add a line to the file config/templates/oaicat.properties of the form:
   Crosswalks.plugin_name=org.dspace.app.oai.PluginCrosswalk
   substituting the plugin's name, e.g. "mets" or "qdc"for plugin_name.
3. Run the bin/install-configs script
4. Restart your servlet container, e.g. Tomcat, for the change to take effect.

DIDL

By activating the DIDL provider, DSpace items are represented as MPEG-21 DIDL objects. These DIDL objects are XML documents that wrap both the Dublin Core metadata that describes the DSpace item and its actual bitstreams. A bitstream is provided inline in the DIDL object in a base64 encoded manner, and/or by means of a pointer to the bitstream. The data provider exposes DIDL objects via the metadataPrefix didl.

The crosswalk does not deal with special characters and purposely skips dissemination of the license.txt file awaiting a better understanding on how to map DSpace rights information to MPEG21-DIDL.

The DIDL Crosswalk can be activated as follows:

• Uncomment the oai.didl.maxresponse item in dspace.cfg
• Uncomment the DIDL Crosswalk entry from the config/templates/oaicat.properties file
• Run the bin/install-configs script
• Restart Tomcat
• Verify the Crosswalk is activated by accessing a URL such as http://mydspace/oai/request?verb=ListRecords&metadataPrefix=didl

Configuring Packager Plugins

Package ingester plugins are configured as named or self-named plugins for the interface org.dspace.content.packager.PackageIngester. Package disseminator plugins are configured as named or self-named plugins for the interface org.dspace.content.packager.PackageDisseminator.

You can add names for the existing plugins, and add new plugins, by altering these configuration properties. See the Plugin Manager architecture for more information about plugins.

Configuring Crosswalk Plugins

Ingestion crosswalk plugins are configured as named or self-named plugins for the interface org.dspace.content.crosswalk.IngestionCrosswalk. Dissemination crosswalk plugins are configured as named or self-named plugins for the interface org.dspace.content.crosswalk.DisseminationCrosswalk.

You can add names for existing crosswalks, add new plugin classes, and add new configurations for the configurable crosswalks as noted below.

Configurable MODS dissemination crosswalk

The MODS crosswalk is a self-named plugin. To configure an instance of the MODS crosswalk, add a property to the DSpace configuration starting with "crosswalk.mods.properties."; the final word of the property name becomes the plugin's name. For example, a property name crosswalk.mods.properties.MODS defines a crosswalk plugin named "MODS".

The value of this property is a path to a separate properties file containing the configuration for this crosswalk. The pathname is relative to the DSpace configuration directory, i.e. the config subdirectory of the DSpace install directory. So, a line like:

 crosswalk.mods.properties.MODS = crosswalks/mods.properties

The MODS crosswalk properties file is a list of properties describing how DSpace metadata elements are to be turned into elements of the MODS XML output document. The property name is a concatenation of the metadata schema, element name, and optionally the qualifier. For example, the contributor.author element in the native Dublin Core schema would be: dc.contributor.author. The value of the property is a line containing two segments separated by the vertical bar ("|"): The first part is an XML fragment which is copied into the output document. The second is an XPath expression describing where in that fragment to put the value of the metadata element. For example, in this property:

dc.contributor.author = <mods:name><mods:role><mods:roleTerm type="text">author</mods:roleTerm></mods:role><mods:namePart>%s</mods:namePart></mods:name> | mods:namePart/text()

For example, given an author named Jack Florey, the crosswalk will insert

<mods:name>
  <mods:role>
    <mods:roleTerm type="text">author</mods:roleTerm>
  </mods:role>
  <mods:namePart>Jack Florey</mods:namePart>
</mods:name> 

Configurable Qualified Dublin Core (QDC) dissemination crosswalk

The QDC crosswalk is a self-named plugin. To configure an instance of the QDC crosswalk, add a property to the DSpace configuration starting with "crosswalk.qdc.properties."; the final word of the property name becomes the plugin's name. For example, a property name crosswalk.qdc.properties.QDC defines a crosswalk plugin named "QDC".

The value of this property is a path to a separate properties file containing the configuration for this crosswalk. The pathname is relative to the DSpace configuration directory, i.e. the config subdirectory of the DSpace install directory. So, a line like:

  crosswalk.qdc.properties.QDC = crosswalks/qdc.properties

You'll also need to configure the namespaces and schema location strings for the XML output generated by this crosswalk. The namespaces property names are of the format:
crosswalk.qdc.namespace.prefix = uri
where prefix is the namespace prefix and uri is the namespace URI.

For example, this shows how a crosswalk named "QDC" would be configured:

crosswalk.qdc.properties.QDC = crosswalks/QDC.properties
crosswalk.qdc.namespace.QDC.dc = http://purl.org/dc/elements/1.1/
crosswalk.qdc.namespace.QDC.dcterms = http://purl.org/dc/terms/
crosswalk.qdc.schemaLocation.QDC  = \
  http://purl.org/dc/terms/ http://dublincore.org/schemas/xmls/qdc/2003/04/02/qualifieddc.xsd

The QDC crosswalk properties file is a list of properties describing how DSpace metadata elements are to be turned into elements of the Qualified DC XML output document. The property name is a concatenation of the metadata schema, element name, and optionally the qualifier. For example, the contributor.author element in the native Dublin Core schema would be: dc.contributor.author. The value of the property is an XML fragment, the element whose value will be set to the value of the metadata field in the property key.

For example, in this property:

  dc.coverage.temporal = <dcterms:temporal />

  <dcterms:temporal>Fall, 2005</dcterms:temporal>

XSLT-based crosswalks

The XSLT crosswalks use XSL stylesheet transformation (XSLT) to transform an XML-based external metadata format to or from DSpace's internal metadata. XSLT crosswalks are much more powerful and flexible than the configurable MODS and QDC crosswalks, but they demand some esoteric knowledge (XSL stylesheets). Given that, you can create all the crosswalks you need just by adding stylesheets and configuration lines, without touching any of the Java code.

A submission crosswalk is described by a configuration key starting with 'crosswalk.submission.", like

  crosswalk.submission.PluginName.stylesheet = path

Here is an example that configures a crosswalk named "LOM" using a stylesheet in [dspace]/config/crosswalks/d-lom.xsl:

  crosswalk.submission.stylesheet.LOM = crosswalks/d-lom.xsl

A dissemination crosswalk is described by a configuration key starting with 'crosswalk.dissemination.", like

  crosswalk.dissemination.PluginName.stylesheet = path

You can make two different plugin names point to the same crosswalk, by adding two configuration entries with the same path, e.g.

    crosswalk.submission.MyFormat.stylesheet = crosswalks/myformat.xslt
    crosswalk.submission.almost_DC.stylesheet = crosswalks/myformat.xslt
 

The dissemination crosswalk must also be configured with an XML Namespace (including prefix and URI) and an XML Schema for its output format. This is configured on additional properties in the DSpace Configuration, i.e.:

   crosswalk.dissemination.PluginName.namespace.Prefix = namespace-URI
   crosswalk.dissemination.PluginName.schemaLocation = schemaLocation value 

   crosswalk.dissemination.qdc.namespace.dc = http://purl.org/dc/elements/1.1/
   crosswalk.dissemination.qdc.namespace.dcterms = http://purl.org/dc/terms/
   crosswalk.dissemination.qdc.schemaLocation = \
      http://purl.org/dc/elements/1.1/ http://dublincore.org/schemas/xmls/qdc/2003/04/02/qualifieddc.xsd 

DSpace Intermediate Metadata (DIM) format

All the elements in a DIM document are in the namespace http://www.dspace.org/xmlns/dspace/dim.

The root element is named dim. It has zero or more children, all field elements. It may have an attribute dspaceType, which identifies the type of object ("ITEM", "COLLECTION", or "COMMUNITY") this metadata describes. This attribute is only guaranteed to be set for dissemination crosswalks.

Each field element may have the following attributes:

• mdschema (Required) The metadata schema, e.g. "dc".
• element (Required) Element name, such as "contributor".
• qualifier Qualifier name, such as "author".
• lang Language code describing language of this entry.

Here is an example of the DIM format:

    <dim:dim xmlns:dim="http://www.dspace.org/xmlns/dspace/dim" dspaceType="ITEM">
      <dim:field mdschema="dc" element="title" lang="en_US">
        The Endochronic Properties of Resublimated Thiotimonline
      </dim:field>
      <dim:field mdschema="dc" element="contributor" qualifier="author">
        Isaac Asimov
      </dim:field>
      <dim:field mdschema="dc" element="language" qualifier="iso">
        eng
      </dim:field>
      <dim:field mdschema="dc" element="subject" qualifier="other" lang="en_US">
        time-travel scifi hoax
      </dim:field>
      <dim:field element="publisher">
        Boston University Department of Biochemistry
      </dim:field>
    </dim:dim>

Testing XSLT Crosswalks

You can test a dissemination crosswalk by hooking it up to an OAI-PMH crosswalk and using an OAI request to get the metadata for a known item.

Testing the submission crosswalk is more difficult, so we have supplied a command-line utility to help. It calls the crosswalk plugin to translate an XML document you submit, and displays the resulting intermediate XML (DIM). Invoke it with:

[dspace]/bin/dsrun org.dspace.content.crosswalk.XSLTIngestionCrosswalk [-l] plugin input-file

Add the -l option to to pass the ingestion crosswalk a list of elements instead of a whole document, as if the List form of the ingest() method had been called. This is needed to test ingesters for formats like DC that get called with lists of elements instead of a root element.

Creating a new Media/Format Filter

Creating a simple Media Filter

New Media Filters must implement the org.dspace.app.mediafilter.FormatFilter interface. More information on the methods you need to implement is provided in the FormatFilter.java source file. For example:

	public class MySimpleMediaFilter implements FormatFilter

Alternatively, you could extend the org.dspace.app.mediafilter.MediaFilter class, which just defaults to performing no pre/post-processing of bitstreams before or after filtering.

	public class MySimpleMediaFilter extends MediaFilter

You must give your new filter a "name", by adding it and its name to the plugin.named.org.dspace.app.mediafilter.FormatFilter field in dspace.cfg. In addition to naming your filter, make sure to specify its input formats in the filter.<class path>.inputFormats config item. Note the input formats must match the short description field in the Bitstream Format Registry (i.e. bitstreamformatregistry table).

	plugin.named.org.dspace.app.mediafilter.FormatFilter = \
  		org.dspace.app.mediafilter.MySimpleMediaFilter = My Simple Text Filter, \
  		...
  	
  	filter.org.dspace.app.mediafilter.MySimpleMediaFilter.inputFormats = Text

If you have a complex Media Filter class, which actually performs different filtering for different formats (e.g. conversion from Word to PDF and conversion from Excel to CSV), you should define this as a Dynamic / Self-Named Format Filter.

Creating a Dynamic or "Self-Named" Format Filter

If you have a more complex Media/Format Filter, which actually performs multiple filtering or conversions for different formats (e.g. conversion from Word to PDF and conversion from Excel to CSV), you should have define a class which implements the FormatFilter interface, while also extending the SelfNamedPlugin class. For example:

	public class MyComplexMediaFilter extends SelfNamedPlugin implements FormatFilter

Since SelfNamedPlugins are self-named (as stated), they must provide the various names the plugin uses by defining a getPluginNames() method. Generally speaking, each "name" the plugin uses should correspond to a different type of filter it implements (e.g. "Word2PDF" and "Excel2CSV" are two good names for a complex media filter which performs both Word to PDF and Excel to CSV conversions).

Self-Named Media/Format Filters are also configured differently in dspace.cfg. Below is a general template for a Self Named Filter (defined by an imaginary MyComplexMediaFilter class, which can perform both Word to PDF and Excel to CSV conversions):

	#Add to a list of all Self Named filters
	plugin.selfnamed.org.dspace.app.mediafilter.FormatFilter = \
  		org.dspace.app.mediafilter.MyComplexMediaFilter
  		
	#Define input formats for each "named" plugin this filter implements
	filter.org.dspace.app.mediafilter.MyComplexMediaFilter.Word2PDF.inputFormats = Microsoft Word
	filter.org.dspace.app.mediafilter.MyComplexMediaFilter.Excel2CSV.inputFormats = Microsoft Excel

As shown above, each Self-Named Filter class must be listed in the plugin.selfnamed.org.dspace.app.mediafilter.FormatFilter item in dspace.cfg. In addition, each Self-Named Filter must define the input formats for each named plugin defined by that filter. In the above example the MyComplexMediaFilter class is assumed to have defined two named plugins, Word2PDF and Excel2CSV. So, these two valid plugin names ("Word2PDF" and "Excel2CSV") must be returned by the getPluginNames() method of the MyComplexMediaFilter class.

These named plugins take different input formats as defined above (see the corresponding inputFormats setting). WARNING: If you neglect to define the inputFormats for a particular named plugin, the MediaFilterManager will never call that plugin, since it will never find a bitstream which has a format matching that plugin's input format(s).

For a particular Self-Named Filter, you are also welcome to define additional configuration settings in dspace.cfg. To continue with our current example, each of our imaginary plugins actually results in a different output format (Word2PDF creates "Adobe PDF", while Excel2CSV creates "Comma Separated Values"). To allow this complex Media Filter to be even more configurable (especially across institutions, with potential different "Bitstream Format Registries"), you may wish to allow for the output format to be customizable for each named plugin. For example:

	#Define output formats for each named plugin
	filter.org.dspace.app.mediafilter.MyComplexMediaFilter.Word2PDF.outputFormat = Adobe PDF
	filter.org.dspace.app.mediafilter.MyComplexMediaFilter.Excel2CSV.outputFormat = Comma Separated Values

Any custom configuration fields in dspace.cfg defined by your filter are ignored by the MediaFilterManager, so it is up to your custom media filter class to read those configurations and apply them as necessary. For example, you could use the following sample Java code in your MyComplexMediaFilter class to read these custom outputFormat configurations from dspace.cfg :

	//get "outputFormat" configuration from dspace.cfg
	String outputFormat = ConfigurationManager.getProperty(MediaFilterManager.FILTER_PREFIX + "." + 
		MyComplexMediaFilter.class.getName() + "." + this.getPluginInstanceName() + ".outputFormat");

Configuration Files for Other Applications

To ease the hassle of keeping configuration files for other applications involved in running a DSpace site, for example Apache, in sync, the DSpace system can automatically update them for you when the main DSpace configuration is changed. This feature of the DSpace system is entirely optional, but we found it useful.

The way this is done is by placing the configuration files for those applications in [dspace]/config/templates, and inserting special values in the configuration file that will be filled out with appropriate DSpace configuration properties. Then, tell DSpace where to put filled-out, 'live' version of the configuration by adding an appropriate property to dspace.cfg, and run [dspace]/bin/install-configs.

Take the apache13.conf file as an example. This contains plenty of Apache-specific stuff, but where it uses a value that should be kept in sync across DSpace and associated applications, a 'placeholder' value is written. For example, the host name:

ServerName @@dspace.hostname@@

The text @@dspace.hostname@@ will be filled out with the value of the dspace.hostname property in dspace.cfg. Then we decide where we want the 'live' version, that is, the version actually read in by Apache when it starts up, will go.

Let's say we want the live version to be located at /opt/apache/conf/dspace-httpd.conf. To do this, we add the following property to dspace.cfg so DSpace knows where to put it:

config.template.apache13.conf = /opt/apache/conf/dspace-httpd.conf

Now, we run [dspace]/bin/install-configs. This reads in [dspace]/config/templates/apache13.conf, and places a copy at /opt/apache/conf/dspace-httpd.conf with the placeholders filled out.

So, in /opt/apache/conf/dspace-httpd.conf, there will be a line like:

ServerName dspace.myu.edu

The advantage of this approach is that if a property like the hostname changes, you can just change it in dspace.cfg and run install-configs, and all of your tools' configuration files will be updated.

However, take care to make all your edits to the versions in [dspace]/config/templates! It's a wise idea to put a big reminder at the top of each file, since someone might unwittingly edit a 'live' configuration file which would later be overwritten.

Browse Index Creation

To create all the various browse indices that you define in the configuration as described in the section Browse Configuration there are a variety of options available to you. You can see these options at any time by running the indexer without any arguments, thus:

[dspace]/bin/dsrun org.dspace.browse.IndexBrowse

This will show you the following options are available to you:

-r,--rebuild

should we rebuild all the indices, which removes old index tables and creates new ones. For use with -f. Mutually exclusive with -d

-s,--start

[-s <int>] start from this index number and work upward (mostly only useful for debugging). For use with -t and -f

-x,--execute

execute all the remove and create SQL against the database. For use with -t and -f

-i,--index

actually do the indexing. Mutually exclusive with -t and -f

-o,--out

[-o <filename>] write the remove and create SQL to the given file. For use with -t and -f

-p,--print

write the remove and create SQL to the stdout. For use with -t and -f

-t,--tables

create the tables only, do not attempt to index. Mutually exclusive with -f and -i

-f,--full

make the tables, and do the indexing. This forces -x. Mutually exclusive with -t and -i

-v,--verbose

print extra information to the stdout. If used in conjunction with -p, you cannot use the stdout to generate your database structure

-d,--delete

delete all the indices, but don't create new ones. For use with -f. This is mutually exclusive with -r

-h,--help

show this help documentation. Overrides all other arguments

The following, then, are examples of what you want to achieve and how this is done with the command line options

Do a full browse re-index, tearing down all old tables and reconstructing with the new configuration

[dspace]/bin/dsrun org.dspace.browse.IndexBrowse -f -r

Do a full browse re-index without modifying the table structure (This should be your default approach if indexing, for example, via a cron job periodically)

[dspace]/bin/dsrun org.dspace.browse.IndexBrowse -i

Destroy and rebuild the database, but do not do the indexing. Output the SQL to do this to the screen and a file, as well as executing it against the database, while being verbose

[dspace]/bin/dsrun org.dspace.browse.IndexBrowse -r -t -p -v -x -o myfile.sql

During installation you will have run the ant target:

ant index

This creates the index tables as per the configuration, and will produce your initial indexed state. From this point on, you should not use ant to generate your indices, as it is not a very good execution environment. Instead, if you feel the need, or your local customisations demand regular full indexing you should set up a regular script to execute:

[dspace]/bin/dsrun org.dspace.browse.IndexBrowse -i