From 1a16a7afdb03d8065a328c44ad1e089ccdf80eda Mon Sep 17 00:00:00 2001 From: Jason Crome Date: Mon, 29 Apr 2024 17:02:47 -0400 Subject: [PATCH 01/10] WIP: release documentation I've clarified some klunkier parts of the existing documentation, and provided some structure for additional documentation. Specifically, I'd like to highlight users and articles (which were not handled by the existing docs). Remaining sections are highlighted with a TODO item. Removed some irrelevant documentation. --- lib/RT/Extension/Import/CSV.pm | 205 +++++++++++++++++++++------------ 1 file changed, 132 insertions(+), 73 deletions(-) diff --git a/lib/RT/Extension/Import/CSV.pm b/lib/RT/Extension/Import/CSV.pm index 502d584..3e66436 100644 --- a/lib/RT/Extension/Import/CSV.pm +++ b/lib/RT/Extension/Import/CSV.pm @@ -1285,7 +1285,36 @@ RT-Extension-Import-CSV =head1 DESCRIPTION -Import data into RT from CSVs. +This extension is used to import data from a comma-separated value +(CSV) file, or any other sort of delimited file, into RT. The importer +provides functionality for importing tickets, transactions, users, and +articles. + +Some common uses of this functionality include: + +=over + +=item Migrating data to RT from another ticketing system (JIRA, ServiceNow, etc.) + +This is the most common method of dumping ticket data from another system. +Whether it be a CSV, TSV, or Excel file, this extension provides the +flexibility needed to get that data into RT. + +=item Syncing data from a non-ticketing system (billing, lead generation, etc.) with RT + +For example, users might create sales leads in a lead-tracking system, +then sync them to RT to create tickets for later follow up and conversation +tracking. + +=item Importing user accounts from another system + +TODO + +=item Importing articles from another knowledge management system (KMS) + +TODO + +=back =head1 RT VERSION @@ -1319,33 +1348,40 @@ Add this line: =head1 CONFIGURATION -The following configuration would be used to import a three-column CSV -of tickets, where the custom field C must be unique. -That option can accept multiple values and the combination of values -must find no existing tickets for insert, or a single ticket for update. -If multiple tickets match, the CSV row is skipped. +The following configuration can import a three-column CSV and illustrates +the basic functionality of the CSV importer: - Set( @TicketsImportUniqueCFs, ('Original Ticket ID') ); + Set( @TicketsImportUniqueCFs, ('Purchase Order ID') ); Set( %TicketsImportFieldMapping, - 'Created' => 'Ticket-Create-Date', - 'CF.Original Ticket ID' => 'TicketID', - 'Subject' => 'name', + 'Created' => 'Ticket-Create-Date', + 'CF.Purchase Order ID' => 'PO-Number', + 'Subject' => 'name', ); +When creating a column mapping, the value to the left of C<=>> is +the RT field name, and to the right is the column name in the CSV +file. CSV files to be imported B have a header line for the +mapping to function. + +In this configuration, the custom field C must be +unique, and can accept accept a combination of values. To insert a row +with this config, RT must find no existing tickets, and for update RT +must only find a single matching row. If neither condition matches, the +CSV row is skipped. + =head2 Excluding Existing Tickets By Status -Some tickets will be opened, issues will be fixed, and the ticket will be marked -as closed. Later, the same asset (e.g., a server) may have a new ticket -opened for a newly found issue. In these cases, a new ticket should be -created and the previous ticket should not be re-opened. To instruct -the importer to exclude tickets in some statuses, set the following option: +In the example above, when searching for an existing ticket for a PO, +it may be necessary to skip certain existing tickets involving this PO +that were previously resolved. To instruct the importer to exclude +tickets in some statuses, set the following option: - Set( @ExcludeStatusesOnSearch, ('fixed')); + Set( @ExcludeStatusesOnSearch, ('resolved', 'cancelled')); =head2 Constant values -If you want to set an RT column or custom field to a static value for -all imported tickets, precede the "CSV field name" (right hand side of +If you want to set an RT column or custom field to the same value for +all imported tickets, precede the CSV field name (right hand side of the mapping) with a slash, like so: Set( %TicketsImportFieldMapping, @@ -1360,7 +1396,7 @@ feature is particularly useful for setting the queue, but may also be useful when importing tickets from CSV sources you don't control (and don't want to modify each time). -=head2 Computed values +=head2 Computed values (advanced) You may also compute values during import, by passing a subroutine reference as the value in the C<%TicketsImportFieldMapping>. This @@ -1378,7 +1414,7 @@ underscores with spaces. ); Using computed columns may cause false-positive "unused column" -warnings; these can be ignored. +warnings during the import; these can be ignored. =head2 Mandatory fields @@ -1386,14 +1422,16 @@ To mark some ticket fields mandatory: Set( @TicketMandatoryFields, 'CF.Severity' ); -Then rows without "CF.Severity" values will be skipped. +In this example, rows without a value for "CF.Severity" values will be +skipped. =head2 Extra Options for Text::CSV_XS -The CSV importer is configured to read the CSV import format determined when initially -testing. However, the Text::CSV_XS module is configurable and can handle different -CSV variations. You can pass through custom options using the configuration below. -Available options are described in the documentation for L. +By default, the importer is configured for a most common variety of text +files (comma-delimited, fields in double quotes). The underlying import +module (L) has many options to handle a wide array of file +options, including unquoted fields, tab-delimited, byte order marking, +etc. To pass custom options to the parser, use the following config: Set( %CSVOptions, ( binary => 1, @@ -1402,6 +1440,69 @@ Available options are described in the documentation for L. escape_char => '`', ) ); +Available options are described in the documentation for L. + +=head2 Special Columns + +=over + +=item Comment or Correspond + +To add a comment or correspond (reply) to a ticket, you can map a CSV column +to "Comment" or "Correspond". When creating a ticket (--insert) you can use +either one and the content will be added to the Create transaction. + +For more information, see the section for L. + +=back + +=head2 TicketsImportTicketIdField + +If the CSV data contains the ids of existing RT tickets, you can set this option +to the name of the column containing the RT ticket id. The importer will then +search for that ticket id and update the ticket data with CSV values. + + Set($TicketsImportTicketIdField, 'RT ticket id'); + +Only one of TicketsImportTicketIdField or @TicketsImportUniqueCFs can be used +for a given CSV file. Also, this option is only valid for --update or --insert-update +modes. You cannot specify the ticket id to be created in --insert mode. + +=head2 TicketTolerantRoles + +By default, if a user can't be loaded via LDAP for a role, like Owner, +the importer will log it and skip creating the ticket. For roles that do not +require a successfully loaded user, set this option with the role name. +The importer will then log the failed attempt to find the user, but still +create the ticket. + + Set(@TicketTolerantRoles, 'CR.Subscribers Peers'); + +=head1 IMPORTING TRANSACTIONS + +The importer can be used to import transactions for existing tickets. +This is useful for bringing the entire ticket history into RT instead +of just the most current ticket data. + +=head2 TransactionsImportFieldMapping + +Set the column mappings for importing transactions from a CSV file. A 'TicketID' mapping +is required for RT to add the transaction to an existing ticket. The 'TicketID' value is +mapped to the custom field 'Original Ticket ID'. + +Attachments can be included by providing the file system path for an attachment. + + Set( %TransactionsImportFieldMapping, + 'Attachment' => 'Attachment', + 'TicketID' => 'SomeID', + 'Created' => 'Date', + 'Type' => 'Type', + 'Content' => 'Content', + 'AttachmentType' => 'FileType' + ); + +=head1 ADVANCED OPTIONS + =head2 Operations before Create or Update The importer provides a callback to run operations before a ticket has been @@ -1470,59 +1571,17 @@ and the type of update, "Create" or "Update". CurrentUser is also passed as it may be needed to call other methods. You can run any code in the callback. It expects no return value. -=head2 Special Columns - -=over - -=item Comment or Correspond - -To add a comment or correspond (reply) to a ticket, you can map a CSV column -to "Comment" or "Correspond". When creating a ticket (--insert) you can use -either one and the content will be added to the Create transaction. - -=back - -=head2 TicketsImportTicketIdField - -If the CSV data contains the ids of existing RT tickets, you can set this option -to the name of the column containing the RT ticket id. The importer will then -search for that ticket id and update the ticket data with CSV values. - - Set($TicketsImportTicketIdField, 'RT ticket id'); - -Only one of TicketsImportTicketIdField or @TicketsImportUniqueCFs can be used -for a given CSV file. Also, this option is only valid for --update or --insert-update -modes. You cannot specify the ticket id to be created in --insert mode. - -=head2 TicketTolerantRoles - -By default, if a user can't be loaded via LDAP for a role, like Owner, -the importer will log it and skip creating the ticket. For roles that do not -require a successfully loaded user, set this option with the role name. -The importer will then log the failed attempt to find the user, but still -create the ticket. - - Set(@TicketTolerantRoles, 'CR.Subscribers Peers'); +=head1 RUNNING THE IMPORT WITH A NON-DEFAULT CONFIGURATION -=head2 TransactionsImportFieldMapping +=head1 EXAMPLES -Set the column mappings for importing transactions from a CSV file. A 'TicketID' mapping -is required for RT to add the transaction to an existing ticket. The 'TicketID' value is -mapped to the custom field 'Original Ticket ID'. +=head2 Import an Excel file -Attachments can be included by providing the file system path for an attachment. - - Set( %TransactionsImportFieldMapping, - 'Attachment' => 'Attachment', - 'TicketID' => 'SomeID', - 'Created' => 'Date', - 'Type' => 'Type', - 'Content' => 'Content', - 'AttachmentType' => 'FileType' - ); +TODO +=head2 Import a tab-separated value (TSV) file -=cut +TODO =head1 AUTHOR From 24ae9b7bfbf13bc5721a74e081e2b87ed8b6707c Mon Sep 17 00:00:00 2001 From: Jason Crome Date: Fri, 3 May 2024 13:32:33 -0400 Subject: [PATCH 02/10] Added example for importing from Excel --- lib/RT/Extension/Import/CSV.pm | 35 +++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/lib/RT/Extension/Import/CSV.pm b/lib/RT/Extension/Import/CSV.pm index 3e66436..c0ae0b7 100644 --- a/lib/RT/Extension/Import/CSV.pm +++ b/lib/RT/Extension/Import/CSV.pm @@ -733,6 +733,7 @@ sub _run_tickets { } } + my $invalid; my $ticket = RT::Ticket->new( $args{CurrentUser} ); my $current_user = $args{CurrentUser}; my %args; @@ -1577,7 +1578,39 @@ in the callback. It expects no return value. =head2 Import an Excel file -TODO +Create a file in Excel, choose B from the menu, and select +C from the B dropdown. Save +it to a file named F. Do not change any additional +options. + +Create a new file called F with the following: + + Set($TicketsImportTicketIdField, 'Ticket No'); + + # RT fields -> Excel columns + Set( %TicketsImportFieldMapping, + 'id' => 'Ticket No', + 'Owner' => 'Assigned To', + 'Status' => 'Status', + 'Subject' => 'Title', + 'Queue' => \'General', + ); + + # Default Excel export options + Set( %CSVOptions, ( + binary => 1, + sep_char => ',', + quote_char => '', + escape_char => '', + ) ); + +Then run the import: + + /opt/rt5/local/plugins/RT-Extension-Import-CSV/bin/rt-extension-import-csv \ + --type ticket \ + --config ExcelImport.pm \ + --insert-update \ + my-excel-test.csv =head2 Import a tab-separated value (TSV) file From 74eabc808dc90ab101ca53e8e0d19d0e378eed82 Mon Sep 17 00:00:00 2001 From: Jason Crome Date: Thu, 9 May 2024 20:13:34 -0400 Subject: [PATCH 03/10] Document use of specific config files/import types --- lib/RT/Extension/Import/CSV.pm | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/lib/RT/Extension/Import/CSV.pm b/lib/RT/Extension/Import/CSV.pm index c0ae0b7..e91ff90 100644 --- a/lib/RT/Extension/Import/CSV.pm +++ b/lib/RT/Extension/Import/CSV.pm @@ -1574,6 +1574,18 @@ in the callback. It expects no return value. =head1 RUNNING THE IMPORT WITH A NON-DEFAULT CONFIGURATION +You can explicitly pass a configuration file to the importer. This is +often used in conjunction when specifying an import type other than +ticket. Use the C<--config> option to specify the path and filename +to the configuration file to use; C<--type> indicates the type of +import to run (article, ticket, transation, or article): + + rt-extension-csv-importer --config /path/to/config.pm --type user /path/to/user-data.csv + rt-extension-csv-importer --config /path/to/config.pm --type ticket /path/to/ticket-data.csv + rt-extension-csv-importer --config /path/to/config.pm --type ticket --update /path/to/ticket-data.csv + rt-extension-csv-importer --config /path/to/config.pm --type transaction /path/to/transaction-data.csv + rt-extension-csv-importer --config /path/to/config.pm --type article --article-class 'VM-Assessment' /path/to/article-data.csv + =head1 EXAMPLES =head2 Import an Excel file From 1aae23486032f83ba2f2f76411e92f66ae8cbde4 Mon Sep 17 00:00:00 2001 From: Jason Crome Date: Fri, 10 May 2024 10:42:17 -0400 Subject: [PATCH 04/10] Add example of importing users --- lib/RT/Extension/Import/CSV.pm | 40 +++++++++++++++++++++++++++++++++- 1 file changed, 39 insertions(+), 1 deletion(-) diff --git a/lib/RT/Extension/Import/CSV.pm b/lib/RT/Extension/Import/CSV.pm index e91ff90..0a21c4b 100644 --- a/lib/RT/Extension/Import/CSV.pm +++ b/lib/RT/Extension/Import/CSV.pm @@ -1309,7 +1309,9 @@ tracking. =item Importing user accounts from another system -TODO +In the above lead generation example, having the same users in both systems +may be convenient. Exporting users from that system and importing them into +RT reduces the amount of administrative work necessary to make that happen. =item Importing articles from another knowledge management system (KMS) @@ -1628,6 +1630,42 @@ Then run the import: TODO +=head2 Import users from another system + +An example application exports users to the following file (F): + + Login,Name,Email,Where At + support_user,Generic Support User,support_user@example.com,Call Center + admin_user,Generic Admin User,admin_user@example.com,HQ + end_user,Generic End User,end_user@example.com,Production Floor + +If you wanted to import those users into RT, create a new file called +F containing the following: + + Set( %UsersImportFieldMapping, + 'Name' => 'Login', + 'RealName' => 'Name', + 'EmailAddress' => 'Email', + 'UserCF.Location' => 'Where At', + ); + + Set( %CSVOptions, ( + binary => 1, + sep_char => ',', + quote_char => '', + escape_char => '', + ) ); + +(this assumes you have created a User Custom Field named Location) + +Then run the following: + + /opt/rt5/local/plugins/RT-Extension-Import-CSV/bin/rt-extension-import-csv \ + --type user \ + --config UserImport.pm \ + --insert \ + users.csv + =head1 AUTHOR Best Practical Solutions, LLC Emodules@bestpractical.comE From a6b6441b5a462e2b4a1a576aff315cebd90f0a21 Mon Sep 17 00:00:00 2001 From: Jason Crome Date: Fri, 10 May 2024 11:38:08 -0400 Subject: [PATCH 05/10] Add example of importing articles --- lib/RT/Extension/Import/CSV.pm | 40 +++++++++++++++++++++++++++++++++- 1 file changed, 39 insertions(+), 1 deletion(-) diff --git a/lib/RT/Extension/Import/CSV.pm b/lib/RT/Extension/Import/CSV.pm index 0a21c4b..7a13017 100644 --- a/lib/RT/Extension/Import/CSV.pm +++ b/lib/RT/Extension/Import/CSV.pm @@ -1315,7 +1315,10 @@ RT reduces the amount of administrative work necessary to make that happen. =item Importing articles from another knowledge management system (KMS) -TODO +RT allows you to include article content in comments and correspondence. +An organization may have a library of this content already available. By +exporting that content and importing it into RT, you can easily include +it on tickets without having to copy/paste from a KMS. =back @@ -1666,6 +1669,41 @@ Then run the following: --insert \ users.csv +=item Importing articles + +An example knowledge management system contains articles your organization +would like to include on RT tickets. The export is delivered as such: + + Title,Synopsis,Content + "Reset Password,"How to Reset a Password","This article explains how to reset a password in detail" + "Create User","How to Create a New User","Instructions on how to create a new user, in excruciating detail" + +Since there are commas in the content, fields in this CSV need to be +quoted, so this needs to be accounted for in the import configuration. +Create F with the following: + + Set( %ArticlesImportFieldMapping, + 'Name' => 'Title', + 'Summary' => 'Synopsis', + 'Content' => 'Content', + ); + + Set( %CSVOptions, ( + binary => 1, + sep_char => ',', + quote_char => '"', + escape_char => '', + ) ); + +You need to add C<--article-class> when running the import: + + /opt/rt5/local/plugins/RT-Extension-Import-CSV/bin/rt-extension-import-csv \ + --type article \ + --article-class General \ + --config ArticleImport.pm \ + --insert \ + articles.csv + =head1 AUTHOR Best Practical Solutions, LLC Emodules@bestpractical.comE From e2ee08616ddedb3ea44b1f84e243aa72e458f3ab Mon Sep 17 00:00:00 2001 From: Jason Crome Date: Fri, 10 May 2024 13:22:12 -0400 Subject: [PATCH 06/10] Add example of importing TSV files --- lib/RT/Extension/Import/CSV.pm | 33 ++++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-) diff --git a/lib/RT/Extension/Import/CSV.pm b/lib/RT/Extension/Import/CSV.pm index 7a13017..ce82716 100644 --- a/lib/RT/Extension/Import/CSV.pm +++ b/lib/RT/Extension/Import/CSV.pm @@ -1631,7 +1631,38 @@ Then run the import: =head2 Import a tab-separated value (TSV) file -TODO +To generate a sample TSV file, select B from +your RT menu. Pick some criteria, and don't change the default display +format or column selections. Click B. On the +resulting search result page, select the B option. + +The following configuration (saved as F) should match the +resulting TSV file: + + Set($TicketsImportTicketIdField, 'id'); + + Set( %TicketsImportFieldMapping, + 'Queue' => \'General', + ); + + Set( %CSVOptions, ( + binary => 1, + sep_char => "\t", + quote_char => '', + escape_char => '', + ) ); + +The double-quotes match the interpolated tab value, rather than a literal +C<\t>. Other columns automatically align with fields in RT, so no +additional mapping is required. + +Importing is similar to the previous example: + + /opt/rt5/local/plugins/RT-Extension-Import-CSV/bin/rt-extension-import-csv \ + --type ticket \ + --config TabImport.pm \ + --insert-update \ + Results.tsv =head2 Import users from another system From 9e55aa8d4f565114c596889b4b7f0203a6245615 Mon Sep 17 00:00:00 2001 From: Jason Crome Date: Fri, 10 May 2024 13:23:01 -0400 Subject: [PATCH 07/10] Update copyright year --- lib/RT/Extension/Import/CSV.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/RT/Extension/Import/CSV.pm b/lib/RT/Extension/Import/CSV.pm index ce82716..3776d52 100644 --- a/lib/RT/Extension/Import/CSV.pm +++ b/lib/RT/Extension/Import/CSV.pm @@ -1752,7 +1752,7 @@ href="http://rt.cpan.org/Public/Dist/Display.html?Name=RT-Extension-Import-CSV"> =head1 LICENSE AND COPYRIGHT -This software is Copyright (c) 2021 by Best Practical LLC +This software is Copyright (c) 2024 by Best Practical LLC This is free software, licensed under: From 3cecb573ee10e110b0c54995e4e7d34dea06506e Mon Sep 17 00:00:00 2001 From: Jason Crome Date: Fri, 10 May 2024 13:44:33 -0400 Subject: [PATCH 08/10] Make the description reflect all functionality We import more than CSVs, and more than just tickets. The description in the docs should reflect that. --- bin/rt-extension-import-csv.in | 22 +++++++----- lib/RT/Extension/Import/CSV.pm | 66 +++++++++++++++++++++++++++++++--- 2 files changed, 74 insertions(+), 14 deletions(-) diff --git a/bin/rt-extension-import-csv.in b/bin/rt-extension-import-csv.in index c6f5579..dab8f1c 100755 --- a/bin/rt-extension-import-csv.in +++ b/bin/rt-extension-import-csv.in @@ -189,7 +189,7 @@ __END__ =head1 NAME -rt-extension-csv-importer - Import data and create tickets in RT +rt-extension-csv-importer - Import data into RT =head1 SYNOPSIS @@ -201,8 +201,8 @@ rt-extension-csv-importer - Import data and create tickets in RT =head1 DESCRIPTION -This script will import/update RT from data in a CSV file. See -L for configuration. +This script will import/update RT from data in a CSV and other types of +delimted files. See L for configuration. =head1 OPTIONS @@ -217,11 +217,11 @@ that may get merged with per-import settings. This option is required. =item C<--type> I or C<-t> I -Specify which type of data we shall import +Specify which type of data shall be imported. This is required. =item C<--article-class> I
-Specify the article class when type is article. +Specify the article class. Required when type is article. =item C<--update> @@ -251,13 +251,17 @@ ticket if not found. By default, tickets containing not-existing users will be skipped; with this flag, they will be created without those users. -This also allows the script to continue processing even errors are found -when parsing csv. +This also allows the script to continue processing even if errors are found +when parsing the CSV file. =item C<--run-scrips> -By default, scrips are temporarily deactivated on C creation. Use -this flag to allow RT to run scrips accordingly. +When importing transactions, the default behavior is to deactivate scrips +on transaction creation. Use this flag to allow RT to run scrips +accordingly. + +This flag only affects transaction import; scrips are always run for +all other import types. =item C<--mdy>, C<--dmy> diff --git a/lib/RT/Extension/Import/CSV.pm b/lib/RT/Extension/Import/CSV.pm index 3776d52..e2fc0db 100644 --- a/lib/RT/Extension/Import/CSV.pm +++ b/lib/RT/Extension/Import/CSV.pm @@ -1050,6 +1050,8 @@ sub load_or_create_user { my $class = shift; my %args = @_; my $user = RT::User->new( delete $args{CurrentUser} ); + $user->Load( $args{EmailAddress} ); + return $user if $user->id; $user->LoadByEmail( $args{EmailAddress} ); return $user if $user->id; @@ -1322,6 +1324,13 @@ it on tickets without having to copy/paste from a KMS. =back +This guide explains how to configure the import tool, and includes +examples of how to run the import with different options. The actual +import is run by L - there is no web-based +component for the import process. Please see the documentation for +L for more in-depth documentation about +the options that the importer can be run with. + =head1 RT VERSION Works with RT 5. @@ -1422,6 +1431,42 @@ underscores with spaces. Using computed columns may cause false-positive "unused column" warnings during the import; these can be ignored. +=head2 Dates and Date Formatting + +When importing tickets, the importer will automatically populate Created +for you, provided there isn't a column in the source data already +mapped to it. Other date fields must be provided in the source data. + +The importer expects incoming date values to conform to L +datetime format (yyyy-mm-dd hh:mm::ss and other accepted variants). If +your source data can't produce this formatting, Perl can help you out. + +For example, if the source data has dates in C format, we +can write a function to append a default time to produce an ISO-formatted +result: + + Set( %TicketsImportFieldMapping, + 'id' => 'Ticket No', + 'Owner' => 'Assigned To', + 'Status' => 'Status', + 'Subject' => 'Title', + 'Queue' => \'General', + 'CF.Delivery Date' => sub { return $_[0]->{ 'Delivery Date' } . ' 00:00:00'; }, + ); + +If you have other date columns you'd like to default to the date/time +the import was run, Perl can help out there, too: + + use POSIX qw(strftime); + Set( %TicketsImportFieldMapping, + 'id' => 'Ticket No', + 'Owner' => 'Assigned To', + 'Status' => 'Status', + 'Subject' => 'Title', + 'Queue' => \'General', + 'CF.Project Start' => sub { return strftime "%Y-%m-%d %H:%M:%S", localtime; } + ); + =head2 Mandatory fields To mark some ticket fields mandatory: @@ -1452,6 +1497,18 @@ Available options are described in the documentation for L option to specify the path and filename -to the configuration file to use; C<--type> indicates the type of -import to run (article, ticket, transation, or article): +You must explicitly pass a configuration file to the importer. Use the +C<--config> option to specify the path and filename to the +configuration file to use; C<--type> indicates the type of import to +run (article, ticket, transation, or article): rt-extension-csv-importer --config /path/to/config.pm --type user /path/to/user-data.csv rt-extension-csv-importer --config /path/to/config.pm --type ticket /path/to/ticket-data.csv From 94a20bbf2fa665876000dc6105ffe93d384c981a Mon Sep 17 00:00:00 2001 From: Jason Crome Date: Fri, 10 May 2024 14:13:29 -0400 Subject: [PATCH 09/10] TicketTolerantRoles are not particular to LDAP Clarified the documentation to reflect this. --- lib/RT/Extension/Import/CSV.pm | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/lib/RT/Extension/Import/CSV.pm b/lib/RT/Extension/Import/CSV.pm index e2fc0db..d8fc02a 100644 --- a/lib/RT/Extension/Import/CSV.pm +++ b/lib/RT/Extension/Import/CSV.pm @@ -1533,13 +1533,13 @@ modes. You cannot specify the ticket id to be created in --insert mode. =head2 TicketTolerantRoles -By default, if a user can't be loaded via LDAP for a role, like Owner, -the importer will log it and skip creating the ticket. For roles that do not -require a successfully loaded user, set this option with the role name. -The importer will then log the failed attempt to find the user, but still -create the ticket. +By default, if a user can't be loaded for a role, like Owner, the importer +will log it and skip creating the ticket. For roles that do not require a +successfully loaded user, set this option with the role name. The importer +will then log the failed attempt to find the user, but still create the +ticket. - Set(@TicketTolerantRoles, 'CR.Subscribers Peers'); + Set(@TicketTolerantRoles, 'CR.Customer'); =head1 IMPORTING TRANSACTIONS From e5b643275546c12b997e90b32488926eb17d13d0 Mon Sep 17 00:00:00 2001 From: Jason Crome Date: Tue, 14 May 2024 08:50:46 -0400 Subject: [PATCH 10/10] Make separate import config files optional For regular import jobs, RT admins may wish to keep these settings alongside the rest of their RT config instead of managing separate config files. --- bin/rt-extension-import-csv.in | 7 +------ lib/RT/Extension/Import/CSV.pm | 9 +++++---- 2 files changed, 6 insertions(+), 10 deletions(-) diff --git a/bin/rt-extension-import-csv.in b/bin/rt-extension-import-csv.in index dab8f1c..0e465d4 100755 --- a/bin/rt-extension-import-csv.in +++ b/bin/rt-extension-import-csv.in @@ -25,11 +25,6 @@ if ( $opt{help} || !$file ) { exit; } -unless ( $opt{config} ) { - Pod::Usage::pod2usage( "Option config is required" ); - exit; -} - unless ( ( $opt{type} // '' ) =~ /^(?:user|ticket|transaction|article)$/ ) { Pod::Usage::pod2usage( "Invalid type provided, should be one of user/ticket/transaction/article" ); exit; @@ -213,7 +208,7 @@ delimted files. See L for configuration. Provides an explicit extra configuration file which is loaded before any other configuration files. This is useful to provide per-import settings. F should B contain settings for options with hashes -that may get merged with per-import settings. This option is required. +that may get merged with per-import settings. =item C<--type> I or C<-t> I diff --git a/lib/RT/Extension/Import/CSV.pm b/lib/RT/Extension/Import/CSV.pm index d8fc02a..71501af 100644 --- a/lib/RT/Extension/Import/CSV.pm +++ b/lib/RT/Extension/Import/CSV.pm @@ -1636,10 +1636,11 @@ in the callback. It expects no return value. =head1 RUNNING THE IMPORT WITH A NON-DEFAULT CONFIGURATION -You must explicitly pass a configuration file to the importer. Use the -C<--config> option to specify the path and filename to the -configuration file to use; C<--type> indicates the type of import to -run (article, ticket, transation, or article): +You can explicitly pass a configuration file to the importer. This is +often used in conjunction when specifying an import type other than +ticket. Use the C<--config> option to specify the path and filename +to the configuration file to use; C<--type> indicates the type of +import to run (article, ticket, transation, or article): rt-extension-csv-importer --config /path/to/config.pm --type user /path/to/user-data.csv rt-extension-csv-importer --config /path/to/config.pm --type ticket /path/to/ticket-data.csv