Differences

This shows you the differences between two versions of the page.

--- help:uservalidation [2021/06/18 14:42]
Rick [6. Test and troubleshoot]
+++ help:uservalidation [2024/02/09 20:38]
Rick
@@ Line 1: / Line 1: @@
+====== User Customization of the Validation tables ======
+MARC Report version 25x supports user customization of the main validation tables used by Batch mode and in an Edit session.
+If the instructions on this page are followed carefully, the user will be able to replace the validation tables distributed by the program with modifications that include new MARC definitions (as published by LC). Theoretically, the user may also add their own 'definitions' to the table.
+__**Prerequisites**__
+MARC Report must be installed, registered, and run at least once, and then closed.
+If custom validation is still in the beta period (ends Dec. 31, 2021), then the [[help:betaoption|beta activation option code]] must be set.
+These instructions require a very good knowledge of MARC21, and at least an average knowledge of Windows and a program like Excel. One should not attempt to modify the tables without this knowledge, as a failed customization will prevent the validation module from loading at all.
+**__Overview__**
+  - Create folders
+  - Copy the TMQ validation table
+  - Make changes to the table
+  - Compile the table
+  - Enable custom validation in MARC Report
+  - Test the customization
+===== 1. Create folders =====
+Two new folders will be required to support customization of the validation tables. One folder is required by MARC Report; the other folder is where you will save your changes.
+----
+\\
+First, create a folder named **validation** in your "Documents\MarcReport" folder. If you are not sure where your "MarcReport" folder is located, [[help:marcreport_folder|follow the steps in this wiki FAQ]] to find it.
+For example, if your username is 'jsmith', this "MarcReport" folder would probably be named:
+  c:\users\jsmith\Documents\MarcReport
+The folder to be created is thus:
+  c:\users\jsmith\Documents\MarcReport\validation
+Do __not__ add any files to this folder. This folder must be used __only__ by the utility descibed in Step 4 below.
+----
+\\
+Next, create a folder for your validation customizations. You will use this folder for all of your customization work (as described in Step 3 below).
+You can name this folder whatever you wish. For example (continuing with jsmith), you might name this folder:
+  c:\users\jsmith\Documents\validation-work
+Do not put this folder in "Documents\MarcReport" as above, but do make sure that it is part of your daily backup routine.
+===== 2. Copy the TMQ validation table(s) =====
+Beginning with version 25x, the raw data used by our validation tables will be distributed in the MARC Report installation folder. The default location for the installation is:
+  C:\Program Files (x86)\TMQ\MARC Report
+If you cannot find your installation folder, [[help:installdir|follow the steps in this wiki FAQ]].
+Three validation tables are distributed in the subdirectory '\defaults\validation'
+(eg. C:\Program Files (x86)\TMQ\MARC Report\defaults\validation)
+The tables are dubbed: 'bib' (bibliographic format), 'auth' (authorities format), and 'hold' (holdings format). Each file is available in two formats: tab-delimited text format ('.txt'), and an older Excel format ('.xls') ((Note: an additional support file for each table--tagged "-matrix"--may also be present. Ignore these files)).
+The naming convention for these tables includes the date that the data was produced on (201202 in the examples); thus, the distributed validation files might be named like:
+  _bib201202.txt
+  _bib201202.xls    9178 rows
+  _auth201202.txt
+  _auth201202.xls   3556 rows
+  _hold201202.txt
+  _hold201202.xls   1974 rows
+The number of rows in each table listed above are as of MARC Report version 256.
+Copy the validation table(s) that you want to customize to your validation work folder. For example, if you want to customize the bib table, copy these files to your work folder:
+  _bib201202.txt
+  _bib201202.xls
+Note: Do not **move** the tables. MARC Report may not work if they are not present in the distribution folder.
+===== 3. Make changes to the table(s) =====
+Open a validation table in excel, make your changes to it, and then save the table.
+It sounds simple, but this will be the most difficult step in customizing the validataion tables.
+The documentation for actually editing the table appears on a separate page. \\
+Visit [[help:validation_tables|Validation Table Format]] for a detailed description of each column in the table and how to enter data for it, including examples.
+After opening the validation table in excel, make sure gridlines are enabled, and that column headings remain visible while scrolling.
+When saving the changes you make to the table:
+  - Be sure to change the filename
+  - The output file must be saved as type: **Text (Tab-delimited) (*.txt)**
+When changing the filename, do not change the beginning tags: '_bib', '_auth', '_hold'.
+Any format other than 'Text (Tab-delimited)' will fail to compile in the next step. (Note: if you are using a program like OpenOffice, an extra step might be needed to create a tab-delimited text file).
+However, it does make sense to additionally save a copy of the modified table in Excel format so that you do not have to re-import it from text whenever you want to change it.
+The Library of Congress typically publishes a one-page summary whenever a new MARC21 version is released. We have always found this summary to be a convenient worksheet to use when updating the validation tables. You can find these update summaries here: https://www.loc.gov/marc/status.html. You can also keep abreast of the future changes that might be coming your way by following the MARC Proposals and MARC Discussion Papers (https://www.loc.gov/marc/mac/index.html). Fortunately, the MARC21 updates that require updates to the validation tables are typically limited to a few entries for each biannual release.
+**IMPORTANT**
+Our validation tables are maintained in Microsoft® Excel®, and any changes to them should also be made in Excel or in an Excel-like program.
+Take care if you import the tab-delimited '.txt' version of the table into Excel (or into an Excel-like program). When one drags and drops a text table onto a worksheet, Excel will seem to import the column and row structure perfectly--however, all leading zeroes are truncated (so that '001' becomes just '1'). This results in a corrupt table as far as MARC Report is concerned. The workaround is to use the **File|Open** menu in Excel to drop into the //Text Import Wizard// ([[help:excelimport|an illustration of such an import may be found on this page]]).
+On the other hand, if you work with the distributed .xls table instead, you won't have to deal with this problem, as all of the columns have already been defined as 'Text'.
+===== 4. Compile the table =====
+Distributed with this version of MARC Report is a utility named **valmaker.exe**, which you should find in the program installation folder (described in Step 2, above). This utility must be run to compile to the customized validation table(s) created in step 3 above.
+Locate this utility and make a shortcut to it. You can drag the shortcut to your validation work folder if you wish.
+Note: do not copy the utility itself--//valmaker// must be launched from a shortcut that points to the MARC Report installation folder.
+Double-click on the shortcut to launch //valmaker//.
+In general, you fill out the form that appears, and then press the "Compile" button. For specific details about using //valmaker.exe//, see the Help in the utility itself.
+//valmaker// performs three functions. First, it applies a matrix of known MARC data elements to your customized table. The purpose of this is to check that the customized validation data makes some sort of sense--a validation of the validation table, so to speak.
+Second, //valmaker// assigns a version number to the table based on the current timestamp. This version number is added as the first row in every validation table--it may help you keep track of different validation files over time.
+The third function is to compile the table into the pseudo-code recognized by MARC Report. This step both compresses (by a ratio  of about 100:1) and encodes the validation table.((Compilation like this was essential when MAC Report was first released, as all of the program code and its data had to fit on a single floppy disk, and the amount of computer memory available was similarly limited.))
+If compilation succeeds, a file called 'valdata.xxx' will be created--where the '.xxx' file extension represents the type of table, as follows:
+  Input    Compiled Output     Type of table
+  _bib     valdata.loc         bibliographic format
+  _auth    valdata.aut         authorities format
+  _hold    valdata.hld         holdings format
+If there are no errors when running //valmaker//, all of the files necessary to initialize the MARC Report validation module are moved from your work folder to your MARC Report **validation** folder.
+===== 5. Enable custom validation =====
+The last step is to start MARC Report, go to the main options menu, and open the Validation tab.
+On the top right, in the box labelled "Validation Tables", select the option labelled **Custom validation ...**.
+{{:help:customvalopt.jpg|}}
+The program will then check to see if a **validation** folder is present, and if so, test whatever validation tables are found in the folder.
+If everything checks out, you will be greeted with a pop-up listing the filename and version number of each validation table that was found.
+If there any errors, the program will uncheck the **Custom validation** box and pop-up a message about the problem it found. Files that trigger error messages should be removed from the **validation** folder.
+If __any__ errors are reported, MARC Report will fallback to using the distributed tables--i.e., it will ignore __all__ of the files in the **validation** folder, even if some are OK.
+__Notes__
+Not every validation table is available for customization. For example, the OCLC Bibliographic table ((this is not a complete validation table, but a minor supplement to the MARC21 Bibliographic table)), and the MARC21 Community Information table((support for this table may be deprecated in future versions of MARC Report)), are not customizable.
+Also, the "Local Validation Data" option (on the right, outside of the screenshot) is incompatible with "Custom validation"--only one of these two options may be enabled at a time. A workaround might be to add any local validation requirements to one of the customizable validation tables.
+===== 6. Test and troubleshoot =====
+If customized validation is successfully activated, the next step is to make a test MARC record that contains all of the content designators that were added to, modified, or deleted from, the customized table. Load that record into MARC Report and test that your changes are working as designed.
+For example, if you have added a new tag (as in the "Example" wiki page), test that
+  - content designators you defined do __not__ trigger an error when they are used, and that
+  - content designators not defined __do__ trigger an error when they are used
+  - MARC Help is working as desired by putting the cursor on your new tag and pressing F1
+Also, don't forget to test for Tag and Subfield repeatability if applicable.
+If any of the testing fails, go back to Step 3 and make the necessary changes to your customized validation table, then repeat the remaining steps described above.
+If, at any time, you end up with a broken validation module, go back to Step 5 and uncheck the **Custom validation** box. Then restart the program. MARC Report should then revert to using the default validation tables.
+If you are not able to get into the program options at all: make sure the program isnt running, open the **marcreport.ini** file in a text editor, and find the line that reads:
+  PREFERUSERVALIDATIONTABLES=Y
+Delete that line, save the .ini file, and restart the program. MARC Report will then revert to the last distributed validation tables, and the validation module should start working as before.

help/uservalidation.txt · Last modified: 2024/02/09 20:38 by Rick

Back to top