John's Vademecum

Try to learn something about everything, and everything about something -Thomas Huxley “Darwin's bulldog” (1824-1895)

User Tools

Site Tools

You are here: My digital Commonplace Book » Public Area » 01 : Radio » 2024 Radio Topics » YaDDNet Data Backup
Trace: • DM41X Curve Fitting • Programs • Test Node 481482 • QSO Logging • Public Area • sidebar • DM41X HP16C Module
public:radio:2024:yaddnet_data_backup

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
public:radio:2024:yaddnet_data_backup [19/04/24 09:53 BST] – [UDP Listener] johnpublic:radio:2024:yaddnet_data_backup [06/03/25 06:49 GMT] (current) – external edit 127.0.0.1
Line 2: Line 2:
  
  
-====== Yaddnet Data Backup ======+====== YaDDNet Data Backup ======
  
 ** Offsite Text file backups of all DSC messages ** ** Offsite Text file backups of all DSC messages **
  
-===== YaDDNet UDP Mirror =====+  * Alan Spindel initially suggested an offsite backup of the DSC messages in 2019 
 +  * YaDDNet has had the facility to send copies of each new message out to specified IP addresses via UDP since that time 
 +  * The DSC messages are sent by UDP as simple plain text, for easy archiving 
 +  * The archived text file can be used if necessary to rebuild the SQL database table containing the DSC messages in case of disaster 
 +  * Since I'm now hosting YaDDNet again I decided I should run a UDP log receiver at home to improve the backup provision 
 +  * The daily archive files will be ''gzipped'' and stored on Dropbox 
 +    * I could share the link to anyone who is interested in using the saved data
  
 +===== YaDDNet UDP Send to Mirror =====
 +
 +  * In the ''PyYadd.py'' function a DSC message is created from the raw incoming data from YaDD (and DSC Decoder) in a standard format.
 +  * This involves converting MMSI -> Coast or Ship name, finding the ''[MID]'' and corresponding country etc.
 +  * Prior to using the newly created message to inject new data into the SQL server ''PyYadd.py'' calls the function ''send_to_mirror()'' to send out the UDP packet to any configured IP addresses
 +  * ''PyYadd.py'' then inserts the new data in the SQL server 
 +  * Any addressed & listening UDP clients receive the UDP packets and append them to a log file
  
 <code python> <code python>
Line 30: Line 43:
  
 #!/usr/bin/env python #!/usr/bin/env python
 +#
 +# udp_logger.py
 +#
  
 import SocketServer import SocketServer
Line 85: Line 101:
  
   * run by Crontab every day at midnight   * run by Crontab every day at midnight
 +  * copies the current running log to a timestamped file
 +  * erases the contents of the running log file 
 +  * ''gzips'' the timestamped copy and moves it to //Dropbox//
  
 <code bash> <code bash>
 +
 +## CRONTAB ENTRY
 +
 00 00 * * * /home/gm4slv/bin/yaddlog_rotate.sh /dev/null 2>&1 00 00 * * * /home/gm4slv/bin/yaddlog_rotate.sh /dev/null 2>&1
 </code> </code>
Line 115: Line 137:
 </code> </code>
  
-   * This creates a ''gzipped'' log file in my Dropbox - for safety - of each day's new DSC messages+   * This creates a ''gzipped'' log file in my Dropbox of each day's new DSC messages
  
 === Starting the UDP Listener at boot-up === === Starting the UDP Listener at boot-up ===
Line 122: Line 144:
  
 <code bash> <code bash>
 +#
 +# CRONTAB ENTRY
 +#
 @reboot python /home/gm4slv/bin/udp_logger.py & @reboot python /home/gm4slv/bin/udp_logger.py &
  
Line 128: Line 153:
  
 --- //John Pumford-Green 19/04/24 09:31 BST// --- //John Pumford-Green 19/04/24 09:31 BST//
 +
 +Last updated : ~~LASTMOD~~
  
  
 ===== Further Information ===== ===== Further Information =====
  
 +[[https://www.yaddnet.org/index.php | YaDDNet]]
  
-{{tag>}}+{{tag>radio yaddnet}}
  
  
public/radio/2024/yaddnet_data_backup.1713516831.txt.gz · Last modified: (external edit)