scores/flux
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge branch 'develop'

Browse Source 
Conflicts:
	README
This commit is contained in:
Michael RICOIS 2017-03-02 14:38:55 +01:00
commit c49610b682
10 changed files with 372 additions and 100 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

246
README Normal file
View File

@ -0,0 +1,246 @@
Mise à disposition fichier de télétransmission
==============================================
Configuration pour les fichiers en entrées
------------------------------------------
'client' => array(
'prestations' => array(
array(
'name' => Nom de la prestation pour référence (UPPERCASE)
'type' => Type du dépot SFTP|FTP
'directory' => Répertoire de dépot (par defaut "send")
'filemask' => Masque du fichier à récupérer ! not working
'in' => array(
--See the list of options--
),
'out' => array(
--See the list of options--
),
),
),
),
Send Options list (in)
----------------------
- FilterName - Filtre par nom de prestation (true) ou par nom défini
- CopyAddDate - Copy file and add a timestamp to the end of the filename
- CopyDeleteAfter - After copying the file delete it in repository
- RunWithEndFile - Only execute action when we receive file with extension .fin or .end
- Log - Log transfert
- Route - For each Route options (cp, ftp, sftp, mail)
cp => realpath of dir
ftp
mail
Recv Options list (out)
-----------------------
- Log - Log read disable by default
Tck Options list (Check Transfert - use with CFT and other transfert mode)
------------------------------------------------
Envoi dans le repo du client fichier *.tck
Recherche du fichier dans la table des flux et remplissage marqueur transfertDate
Crontab - incron
================
Exemple surveillance livraison de fichiers par le client
$fluxBasePath IN_CLOSE_WRITE php /home/batchFlux/fileSend.php --file $@/$# >> /home/flux/shared/send.log 2>&1
Exemple surveillance lecture du fichier mis en dépot
$fluxBasePath IN_CLOSE_NOWRITE,IN_ACCESS,IN_DELETE,IN_NO_LOOP php /home/batchFlux/fileRead.php --file $@/$# >> /home/flux/shared/read.log 2>&1
Exemple surveillance retour par autre programme de transmission
$fluxBasePath IN_CLOSE_WRITE php /home/batchFlux/fileTck.php --file $@/$# >> /home/flux/shared/tck.log 2>&1
Attention, une seule entrée de repertoire peut être saisie dans la incrontab.
fileSend a été prévue pour gérer correctement les extension tck
Sinon préférer l'utilisation de trigger.php
$fluxBasePath IN_CLOSE_WRITE,IN_CLOSE_NOWRITE,IN_ACCESS,IN_DELETE,IN_NO_LOOP /home/batchFlux/trigger.php --file $@/$# --event $%
TODO
====
Créer fileMail => Recupération element par mail, enregistrement
Filtre sur email et/ou sujet
Créer fileTransfert => envoi sur le FTP, SFTP du client
Note CFT
========
PARTNERS
&PART Partner name (ID of CFTPART)
&GROUP Group to which the partner belongs
&SPART Sending partner name
&RPART Receiving partner name
&IPART Intermediate partner name
&NPART Network name of partner sending data (NSPART or NRPART according to the transfer direction)
TRANSFER
&IDT Transfer identifier
&NIDT Protocol transfer identifier
&IDTU Local transfer counter (unique)
&PIDTU Parent idtu of the child transfers
&PHASE Processing phases to help manage transfer flows
&PHASESTEP Step in processing phase
&APPSTATE State step for the processing script to restart if relaunched
&NSUB Counter for the submitting of end-of-transfer procedures, error procedures and procedures submitted by SUBMIT.
If 4 characters long, the counter is reset to 1 after 9999
&DIAGI Internal diagnostic code value
&DIAGP Protocol diagnostic code value
&COMP Compression negotiated for the transfer
&NBT Number of bytes transferred
&PRI Transfer CFT priority for the transfer (0 to 255)
&QQ Number of the day in the year associated with the transfer identifier
&SELFNAME Name of the generic transfer selection file
&FCODE Code for the data in a file
&TRTYPE Available at the end of transfer to designate FILE, MESSAGE, REPLY, or NACK
&NCODE Code for the data sent over the network
&EXITFREE Free communication area between multiple exits
&XLATE Transcoding table used during transfer
&MODE Server mode = ‘S’ transfer, Requester mode = ‘R’ transfer
FILE
&IDF Model file identifier (logical name)
&FNAME Physical file local name
&FKEYLEN Length (received) of the indexed file key at the sender’s site
&FKEYPOS Position (received) of the indexed file key at the sender’s site
&NBR Number of records in the file
&BLKNUM Catalog block number
&XLATE Identifier of the translation table used
&NBC Number of bytes in the transferred file
&NIDF Model file network identifier
&FDB Database name
&FCHARSET Local file encoding
&NCHARSET Destination file encoding for network data
Receiving
&NFNAME Physical file network name
&UNIT Physical file volume name for received file (MVS, z/OS)
&UNITC Physical file unit class for received file (MVS, z/OS)
&PATH Local file path of the received file
&ROOT Local file root for the received file
&SUF Local file suffix for the received file
Sending
&SFNAME Name of file to send
&FUNIT Physical file volume name for sending file (z/OS,MVS)
&FUNITC Physical file unit for sending file (z/OS,MVS)
&FPATH Prefix (file path) of the sending file
&FROOT Root (actual file name) of the sending file
&FSUF Suffix associated with file name of the sending file
DATE and TIME associated with a FILE
&FDATE Date associated with the file
&FTIME Time associated with the file
&FYEAR Year associated with the file
&FMONTH Month associated with the file
&FDAY Day associated with the file
DATE and TIME associated with a CATALOG
&CDATE Catalog entry date
&CTIME Transfer start date
&BTIME Transfer start time
&BYEAR Start year for the transfer
&BMONTH Start month for the transfer
&BDAY Transfer start day
&EDATE Transfer end date
&ETIME Transfer end time f
&EYEAR Transfer end year
&EMONTH Transfer end month
&EDAY Transfer end day
&TT Transmission duration in seconds (TIMES attribute in the Transfer CFT catalog)
CONTROL OUTPUT
&FLOG Name of last log file used by Transfer CFT
&FACCNT Name of last statistics file used by Transfer CFT
&FCAT Name of catalog used by Transfer CFT
Note INOTIFY
============
Utilisation des événements INOTIFY pour la surveillance des dossiers
The command may contain these wildcards:
$$ - a dollar sign
$@ - the watched filesystem path (see above)
$# - the event-related file name
$% - the event flags (textually)
$& - the event flags (numerically)
Events
IN_ACCESS
File was accessed (read) (*).
IN_ATTRIB
Metadata changed, e.g., permissions, timestamps, extended attributes, link count (since Linux 2.6.25), UID, GID, etc. (*).
IN_CLOSE_WRITE
File opened for writing was closed (*).
IN_CLOSE_NOWRITE
File not opened for writing was closed (*).
IN_CREATE
File/directory created in watched directory (*).
IN_DELETE
File/directory deleted from watched directory (*).
IN_DELETE_SELF
Watched file/directory was itself deleted.
IN_MODIFY
File was modified (*).
IN_MOVE_SELF
Watched file/directory was itself moved.
IN_MOVED_FROM
File moved out of watched directory (*).
IN_MOVED_TO
File moved into watched directory (*).
IN_OPEN
File was opened (*).
When monitoring a directory, the events marked with an asterisk (*) above can occur for files in the directory,
in which case the name field in the returned inotify_event structure identifies the name of the file within the
directory.
The IN_ALL_EVENTS macro is defined as a bit mask of all of the above events. This macro can be used as the mask
argument when calling inotify_add_watch(2).
Two additional convenience macros are IN_MOVE, which equates to IN_MOVED_FROM|IN_MOVED_TO, and IN_CLOSE, which
equates to IN_CLOSE_WRITE|IN_CLOSE_NOWRITE.
The following further bits can be specified in mask when calling inotify_add_watch(2):
IN_DONT_FOLLOW (since Linux 2.6.15)
Don't dereference pathname if it is a symbolic link.
IN_EXCL_UNLINK (since Linux 2.6.36)
By default, when watching events on the children of a directory, events are generated for children even after they have been unlinked from the directory. This can result in large numbers of uninteresting events for some applications (e.g., if watching /tmp, in which many applications create temporary files whose names are immediately unlinked). Specifying IN_EXCL_UNLINK changes the default behavior, so that events are not generated for children after they have been unlinked from the watched directory.
IN_MASK_ADD
Add (OR) events to watch mask for this pathname if it already exists (instead of replacing mask).
IN_ONESHOT
Monitor pathname for one event, then remove from watch list.
IN_ONLYDIR (since Linux 2.6.15)
Only watch pathname if it is a directory.
The following bits may be set in the mask field returned by read(2):
IN_IGNORED
Watch was removed explicitly (inotify_rm_watch(2)) or automatically (file was deleted, or file system was unmounted).
IN_ISDIR
Subject of this event is a directory.
IN_Q_OVERFLOW
Event queue overflowed (wd is -1 for this event).
IN_UNMOUNT
File system containing watched object was unmounted.

41
scripts/build/incron.php → buidIncron.php Normal file → Executable file
View File

@ -8,36 +8,41 @@ defined('APPLICATION_ENV')
|| define('APPLICATION_ENV', (getenv('APPLICATION_ENV') ? getenv('APPLICATION_ENV') : 'production'));
// --- Composer autoload
require_once realpath(__DIR__ . '/../../vendor/autoload.php');
require_once realpath(__DIR__ . '/vendor/autoload.php');
// --- Create application, bootstrap, and run
$application = new Zend_Application(APPLICATION_ENV, APPLICATION_PATH . '/configs/application.ini');
$application->bootstrap()->run();
// --- Options
$displayUsage = false;
try {
$opts = new Zend_Console_Getopt(
//Options
array(
'help|?' => "Display usage information.",
'list' => "List client and prestations.",
'generate' => "Generate",
'send=s' => "Generate send incron file [client]/[name]",
'read=s' => "Generate read incron file [client]/[name]",
)
);
$opts = new Zend_Console_Getopt(array(
'help|?' => "Display usage information.",
'list' => "List client and prestations.",
'generate' => "Generate",
'send=s' => "Generate send incron file [client]/[name]",
'read=s' => "Generate read incron file [client]/[name]",
));
$opts->parse();
$optsNb = $opts->getOptions();
} catch (Zend_Console_Getopt_Exception $e) {
echo $e->getUsageMessage();
exit;
$displayUsage = true;
}
//Usage
if(isset($opts->help))
{
// --- Aide / Options
if ($optsNb == 0 || isset($opts->help)) {
$displayUsage = true;
}
// --- Usage
if ($displayUsage) {
echo $opts->getUsageMessage();
exit;
}
$logPath = realpath(__DIR__ . '/../shared');
$prestations = include APPLICATION_PATH . '/../config.php';
if ($opts->list)
@ -83,7 +88,7 @@ if ($opts->generate)
}
$fluxBasePath = '/home/data/' . strtolower($item['type']) . '/' . $client . '/' . $directory;
$file = __DIR__ . '/incron.d/' . strtolower($item['type']) . '_' . $client;
file_put_contents($file, "$fluxBasePath IN_CLOSE_WRITE php /home/flux/current/send.php --file $@/$# >> /home/log/send.log 2>&1");
file_put_contents($file, "$fluxBasePath IN_CLOSE_WRITE php /home/flux/current/send.php --file $@/$# >> $logPath/send.log 2>&1");
echo "File created $file\n";
}
}
@ -109,7 +114,7 @@ if ($opts->generate)
}
$fluxBasePath = '/home/data/' . strtolower($item['type']) . '/' . $client. '/' . $directory;
$file = __DIR__ . '/incron.d/' . strtolower($item['type']) . '_' . $client . '_recv';
file_put_contents($file, "$fluxBasePath IN_ACCESS php /home/flux/current/read.php --file $@/$# >> /home/log/read.log 2>&1");
file_put_contents($file, "$fluxBasePath IN_ACCESS php /home/flux/current/read.php --file $@/$# >> $logPath/read.log 2>&1");
echo "File created $file\n";
}
}

29
clean.php
View File

@ -13,24 +13,27 @@ require_once realpath(__DIR__ . '/vendor/autoload.php');
// --- Create application, bootstrap, and run
$application = new Zend_Application(APPLICATION_ENV, APPLICATION_PATH . '/configs/application.ini');
// --- Options
$displayUsage = false;
try {
$opts = new Zend_Console_Getopt(
//Options
array(
'help|?' => "Displays usage information.",
'cron' => "Mandatory option with cron",
'verbose|v' => "Affichage de ce qui est fait."
)
);
$opts = new Zend_Console_Getopt(array(
'help|?' => "Displays usage information.",
'cron' => "Mandatory option with cron",
'verbose|v' => "Affichage de ce qui est fait."
));
$opts->parse();
$optsNb = $opts->getOptions();
} catch (Zend_Console_Getopt_Exception $e) {
echo $e->getUsageMessage();
exit;
$displayUsage = true;
}
//Usage
if( isset($opts->help) || count($opts->getOptions())==0 )
{
// --- Aide / Options
if ($optsNb == 0 || isset($opts->help)) {
$displayUsage = true;
}
// --- Usage
if ($displayUsage) {
echo "Clean directories.\n";
echo $opts->getUsageMessage();
exit;

3
fallback.php
View File

@ -22,12 +22,13 @@ try {
'verbose|v' => "Affichage de ce qui est fait."
));
$opts->parse();
$optsNb = $opts->getOptions();
} catch (Zend_Console_Getopt_Exception $e) {
$displayUsage = true;
}
// --- Aide / Options
if (count($opts->getOptions())==0 || isset($opts->help)) {
if ($optsNb == 0 || isset($opts->help)) {
$displayUsage = true;
}

27
read.php
View File

@ -17,24 +17,27 @@ require_once realpath(__DIR__ . '/vendor/autoload.php');
// --- Create application, bootstrap, and run
$application = new Zend_Application(APPLICATION_ENV, APPLICATION_PATH . '/configs/application.ini');
// --- Options
$displayUsage = false;
try {
$opts = new Zend_Console_Getopt(
//Options
array(
'help|?' => "Displays usage information.",
'file|f=s' => "Give the full file path to integrate",
'debug' => "Send a mail for debug",
)
);
$opts = new Zend_Console_Getopt(array(
'help|?' => "Displays usage information.",
'file|f=s' => "Give the full file path to integrate",
'debug' => "Send a mail for debug",
));
$opts->parse();
$optsNb = $opts->getOptions();
} catch (Zend_Console_Getopt_Exception $e) {
echo $e->getUsageMessage();
exit;
$displayUsage = true;
}
// --- Aide / Options
if ($optsNb == 0 || isset($opts->help)) {
$displayUsage = true;
}
// --- Usage
if( isset($opts->help) || count($opts->getOptions())==0 )
{
if ($displayUsage) {
echo "Execute basic action when a customer read a file.\n";
echo $opts->getUsageMessage();
exit;

36
recv.php
View File

@ -19,26 +19,29 @@ require_once realpath(__DIR__ . '/vendor/autoload.php');
// --- Create application, bootstrap, and run
$application = new Zend_Application(APPLICATION_ENV, APPLICATION_PATH . '/configs/application.ini');
// --- Options
$displayUsage = false;
try {
$opts = new Zend_Console_Getopt(
//Options
array(
'help|?' => "Displays usage information.",
'cron' => "Mandatory option in crontab",
'file=s' => "Manually define the file to process",
'client=s' => "Define the client name for getting the file manually",
'debug' => "Send a mail for debug",
)
);
$opts = new Zend_Console_Getopt(array(
'help|?' => "Displays usage information.",
'cron' => "Mandatory option in crontab",
'file=s' => "Manually define the file to process",
'client=s' => "Define the client name for getting the file manually",
'debug' => "Send a mail for debug",
));
$opts->parse();
$optsNb = $opts->getOptions();
} catch (Zend_Console_Getopt_Exception $e) {
echo $e->getUsageMessage();
exit;
$displayUsage = true;
}
// --- Aide / Options
if ($optsNb == 0 || isset($opts->help)) {
$displayUsage = true;
}
// --- Usage
if( isset($opts->help) || count($opts->getOptions())==0 )
{
if ($displayUsage) {
echo "Place files in right directory for sending to the customer.\n";
echo $opts->getUsageMessage();
exit;
@ -131,7 +134,8 @@ if ( $result->count() > 0 ) {
}
// --- Copy du fichier
if ( copy($source, $dest) ) {
exec("cp $source $dest", $output, $result);
if ($result == 0) {
// --- Detail du fichier
$nbLines = 0;
@ -146,7 +150,7 @@ if ( $result->count() > 0 ) {
chown($dest, 'ftpuser');
chgrp($dest, 'ftpgroup');
}
elseif($item->depotType == 'SFTP') {
elseif ($item->depotType == 'SFTP') {
chown($dest, $item->client);
}

6
scripts/build/etc/cron.d/flux
View File

@ -1,6 +0,0 @@
# Gestion des flux a envoyer au client
*/15 4-18 * * * root php /home/flux/current/recv.php --cron >> /home/log/recv.log 2>&1
# Nettoyage
01 18 * * * root php /home/flux/current/clean.php --cron >> /home/log/clean.log 2>&1

33
send.php
View File

@ -18,30 +18,35 @@ require_once realpath(__DIR__ . '/vendor/autoload.php');
// --- Create application, bootstrap, and run
$application = new Zend_Application(APPLICATION_ENV, APPLICATION_PATH . '/configs/application.ini');
// --- Options
$displayUsage = false;
try {
$opts = new Zend_Console_Getopt(
// --- Options
array(
'help|?' => "Displays usage information.",
'file|f=s' => "Give the full file path to integrate",
'mail' => "Only send a mail when a file is write",
'debug' => "Send a mail for debug",
)
);
$opts = new Zend_Console_Getopt(array(
'help|?' => "Displays usage information.",
'file|f=s' => "Give the full file path to integrate",
'mail' => "Only send a mail when a file is write",
'debug' => "Send a mail for debug",
));
$opts->parse();
$optsNb = $opts->getOptions();
} catch (Zend_Console_Getopt_Exception $e) {
echo $e->getUsageMessage();
exit;
$displayUsage = true;
}
// --- Aide / Options
if ($optsNb == 0 || isset($opts->help)) {
$displayUsage = true;
}
// --- Usage
if( isset($opts->help) || count($opts->getOptions())==0 )
{
if ($displayUsage) {
echo "Execute basic action when a customer send a file.\n";
echo $opts->getUsageMessage();
exit;
}
$logPath = realpath(__DIR__ . '/../shared');
// --- Get the file
if ( isset($opts->file) )
{
@ -64,7 +69,7 @@ if ( isset($opts->file) )
}
// --- Don't play with *.tck files
if ($extension == 'tck') {
passthru(__DIR__ . '/tck.php --file '.$opts->file.' >> /home/log/tck.log 2>&1');
passthru(__DIR__ . '/tck.php --file '.$opts->file.' >> '.$logPath.'/tck.log 2>&1');
exit;
}
$client = basename(dirname($pathParts['dirname']));

27
tck.php
View File

@ -18,24 +18,27 @@ require_once realpath(__DIR__ . '/vendor/autoload.php');
// --- Create application, bootstrap, and run
$application = new Zend_Application(APPLICATION_ENV, APPLICATION_PATH . '/configs/application.ini');
// --- Options
$displayUsage = false;
try {
$opts = new Zend_Console_Getopt(
//Options
array(
'help|?' => "Displays usage information.",
'file|f=s' => "Give the full file path to integrate",
'debug' => "Send a mail for debug",
)
);
$opts = new Zend_Console_Getopt(array(
'help|?' => "Displays usage information.",
'file|f=s' => "Give the full file path to integrate",
'debug' => "Send a mail for debug",
));
$opts->parse();
$optsNb = $opts->getOptions();
} catch (Zend_Console_Getopt_Exception $e) {
echo $e->getUsageMessage();
exit;
$displayUsage = true;
}
// --- Aide / Options
if ($optsNb == 0 || isset($opts->help)) {
$displayUsage = true;
}
// --- Usage
if( isset($opts->help) || count($opts->getOptions())==0 )
{
if ($displayUsage) {
echo "Mark as transfert.\n";
echo $opts->getUsageMessage();
exit;

24
trigger.php
View File

@ -13,6 +13,8 @@ require_once realpath(__DIR__ . '/vendor/autoload.php');
// --- Create application, bootstrap, and run
$application = new Zend_Application(APPLICATION_ENV, APPLICATION_PATH . '/configs/application.ini');
// --- Options
$displayUsage = false;
try {
// --- Options
$opts = new Zend_Console_Getopt(array(
@ -21,14 +23,18 @@ try {
'event=s' => "Event",
));
$opts->parse();
$optsNb = $opts->getOptions();
} catch (Zend_Console_Getopt_Exception $e) {
echo $e->getUsageMessage();
exit;
$displayUsage = true;
}
// --- Aide / Options
if ($optsNb == 0 || isset($opts->help)) {
$displayUsage = true;
}
// --- Usage
if( isset($opts->help) || count($opts->getOptions())==0 )
{
if ($displayUsage) {
echo "Proxy Event Trigger.\n";
echo $opts->getUsageMessage();
exit;
@ -39,6 +45,8 @@ if (empty($opts->file)) {
exit;
}
$logPath = realpath(__DIR__ . '/../shared');
$pathParts = pathinfo($opts->file);
$filename = $pathParts['basename'];
$extension = $pathParts['extension'];
@ -47,16 +55,16 @@ switch($opts->event)
{
case 'IN_CLOSE_WRITE':
if ($extension == 'tck') {
passthru(__DIR__ . '/tck.php --file '.$opts->file.' >> /home/log/tck.log 2>&1');
passthru(__DIR__ . '/tck.php --file '.$opts->file.' >> '.$logPath.'/tck.log 2>&1');
} else {
passthru(__DIR__ . '/send.php --file '.$opts->file.' >> /home/log/send.log 2>&1');
passthru(__DIR__ . '/send.php --file '.$opts->file.' >> '.$logPath.'/send.log 2>&1');
}
break;
case 'IN_ACCESS':
case 'IN_CLOSE_NOWRITE':
passthru(__DIR__ . '/read.php --file '.$opts->file.' >> /home/log/read.log 2>&1');
passthru(__DIR__ . '/read.php --file '.$opts->file.' >> '.$logPath.'/read.log 2>&1');
break;
case 'IN_DELETE':
passthru(__DIR__ . '/read.php --file '.$opts->file.' >> /home/log/read.log 2>&1');
passthru(__DIR__ . '/read.php --file '.$opts->file.' >> '.$logPath.'/read.log 2>&1');
break;
}