RedZone

From popdata
Jump to: navigation, search
Back to >> Systems
See also >> RLU >> RLU Process and Procedures Manual >> Secure_Upload/Download for instructions to researchers.
  • See Services/transfer for procedure yellowfolder for moving files between Franklin / Cartier2 / SRE-file-servers.

General Info

Active Directory Domain Controller

  • Larsen (larsen.popdata.bc.ca 10.180.10.105)

* VMs on delasalle.popdata.bc.ca (10.180.10.120) and joliet.popdata.bc.ca (10.180.10.125) VMware ESXi servers.

RZ Virtual Machines See Services/KVM#RZ_VM_by_user

File Servers:

  • Franklin (debian linux machine)
    • samba shares:
      • \\franklin\helpred
      • //franklin/transfer for yellow folders
  • George2
    • SAMBA shares:
      • Z: \\george2\USERNAME

Required folders for yellowfolder redzone transfers

  • Cartier: /data/transfer/yellowzone/${USER}
  • Franklin: script /usr/local/sbin/create_yellowfolder_dirs.sh creates user folders under:
    • /home /data/saved/incoming-nondata /data/saved/outgoing-nondata /data/saved/yellowzone /data/transfer/redzone /data/transfer/yellowzone
  • Defuca, George2, Ericsson : in ${HOME}/TRANFER , links REDZONE=>/data/transfer/redzone/${USER} YELLOWZONE=>/data/transfer/yellowzone/bhills
    • /usr/local/sbin/create_yellowfolder_dirs.sh creates under /data/saved/yellowzone (& yellowzone), /data/transfer/redzone (& yellowzone)

Yellow Folders

Running

  • runs on Franklin (RedZone terminal server) - /usr/local/sbin/yellowfolder.pl
    • Start: /etc/init.d/incron start (using /etc/incron.d/yellowfolder)
    • Stop: /etc/init.d/incron stop
  • logs are in /var/log/yellow*
  • files are copied with rsync; rsync daemon on Fraser started in /etc/init.d
  • from/to folders are on Franklin under /data/transfer/ ( SAMBA share "\\popdatarz\transfer");
  • transfer for {USERNAME} triggered by running "GET-IT" on Franklin, which creates a folder .trigger/{USERNAME} under top-level folder (ex: incoming-nondata/.trigger/dlaplante) . The script GET-IT.lnk(windows) or GET_IT.sh(unix) creates this folder for $USER, so to trigger for another user (e.g. lchen) do:
    • USER=lchen ./SEND-IT.sh
    • For incoming-data (Cartier2 -> Franklin), sub-directory is "incoming-data/newdata"
  • corresponding folders on Cartier2 are all under /home/shares/transfer/ (<=/data/transfer)
    • from Cartier2: incoming-data , incoming-nondata
    • to Cartier2: outgoing-nondata , outgoing-notify , outgoing-sftp , outgoing-dl
  • to SRE file-servers (fraser verendrye ...) Fraser: outgoing-sre (-> /home/outgoing-sre/ write-only )
franklin.popdata.bc.ca:/data/transfer/...
    • to or from cartier2.popdata.bc.ca:/home/shares/transfer/...

YellowZone non-data transfers Cartier2 <=> Franklin

  • Email from Jim 2014-04-10 to RedZone users
There are two types of transfers for non-data type files: 
Intra-Redzone (between the defuca/george/ericsson vms) and RedZone<->YellowZone

Instructions:

In your home directory on defuca, ericsson, and george2 there is a
folder TRANSFER.  Inside TRANSER are folders REDZONE and YELLOWZONE:
and some scripts you can double click.

TRANSFER\
   REDZONE\
	SEND-IT-TO-REDZONE
    	GET-IT-FROM-REDZONE
   YELLOWZONE\
       SEND-IT-TO-YELLOWZONE
   	GET-IT-FROM-YELLOWZONE

When you click SEND-IT-TO-REDZONE it will send anything in the REDZONE
folder to a common area.  If you click on GET-IT-FROM-REDZONE it will
retrieve anything from the common area.  This works on all three
redzone areas (prepare/defuca,link/ericsson,extract/george2*)

When you click SEND-IT-TO-YELLOWZONE it will send anything in the
YELLOWZONE folder to \\cartier\transfer\yellowzone\<USER> in the
yellowzone.  Likewise clicking on GET-IT-FROM-YELLOWZONE will retrieve
anything from \\cartier\transfer\yellowzone\<USER>.  When retrieving 
files from the yellowzone, they go through a couple of hops, so they
will take a few seconds to appear.
  • SEND-IT-TO-REDZONE sends from /data/transfer/redzone/${USER} to same folder on Franklin (mkdir /data/transfer/redzone/.trigger-redzone-out/${USER}).
    GET-IT-FROM-REDZONE reverses that.
  • The above two-hop YellowZone to RedZone transfers works from Defuca/George2/Ericsson through Franklin to Cartier2.
    • The trick is that YellowFolder.pl on e.g. George2 sends a trigger directory to Franklin before (or after, as appropriate).
  • To do one-hop transfer between Franklin and Cartier2 (/home/transfer/yellowzone/{USER})
      • cd /data/transfer/yellowzone && mkdir .trigger-yellowzone-in/{USER} (or mkdir .trigger-yellowzone-out/{USER} )
      • -OR- /data/transfer/yellowfolder/SEND_IT_TO_YELLOWZONE.sh [optional: USERNAME]

Firewall

  • Machine: vancouver.popdata.bc.ca (via Mackenzie)
  • fwbuilder feeds set of rules to iptables. things will still work until a reboot
    • Stop firewall: /etc/init.d/fwbuilder stop
      • Reset rules to default: allow outbound connections. RZ network still not accessible from outside.
    • start firefall: /etc/init.d/fwbuilder start . Automatic on reboot.
    • status: /etc/init.d/fwbuilder listfilter (same as /sbin/iptables -L)

Modifying the Firewall

  • go to cabot.popdata.bc.ca as root
  • run fwbuilder (must have X11 session forwarding)
  • load /root/vancouver.fwb
  • change it carefully :)
  • go to Rules->Compile and write out the result to rz.fw
  • place the /root/rz.fw file on cartier in: /data/transfer/incoming-data/newdata
  • go to franklin as root
  • cd /data/transfer/incoming-data; GET-IT.sh
  • cd newdata
  • scp rz.fw jim@vancouver:
  • go to vancouver as root
  • cd /etc/fwbuilder
  • cp vancouver.fw /home/jim/vancouver.fw-2012-01-23
  • cp /home/jim/rz.fw vancouver.fw
  • /etc/init.d/fwbuilder stop
  • /etc/init.d/fwbuilder start