Courses

From popdata
Jump to: navigation, search

Back to SRE (this page was formerly called SRE-courses)

See http://www.popdata.bc.ca/etu Popdata Education and Training Unit

Contents

Free online PopData courses at uVic Continuing Studies with PopData RTL lab access

UVic free courses with self-registration are allowed though VPN (profile "course") with password.

  • See https://training.popdata.bc.ca
    • for example 'HGEO 101' (free registration req) uses r:\Courses\GIS_epi.
      and "Tutorial 1_GIS and Epidemiology.pdf" starts with copying (Save As) BC_LHA map file to VIHA_MAP_[your initials], actually under C:\...\Documents\ARCGIS\ )
  • In fall 2013, all courses were moved from Continuing Studies registration and associated Moodle site to ETU PopData Moodle site and server at PopData for registration and access.
    See Systems/Account_Management#Self-service_RTL_account_creation

Course management Routine

  1. First time, see below for creating a course "group" in PDS Courses#PDS_for_SRTL
  2. Always notify srtl@popdata.bc.ca of upcoming course dates: course name, instructor, first class, first use of SRTL, last use of SRTL, vetting & export of student assignments.
    If you can, update the calendar below. Remind IT staff to review course file permissions.
  3. Enable students: Systems/Account_Management#Process_for_creating_SRTL_course-specific_accounts
    • Distribute Yubikeys after getting signatures
    • Create students' primary and course-specific account.
    • When ready for first login, for each student
      1. activate SRTL vpn access for primary account, and
      2. activate SRTL Windows access for secondary account.
  4. After end of course, use PDS to disable student access. Courses#Expiry_.2F_disabling_of_course_users. For each student
    1. Edit student's course account (ex: TestUser-spph-123) and remove "activated for SRTL". This will prevent SRTL Windows login, and prevent creation (re-creation) of student folders on R: drive.
    2. Edit student's main account (Ex: TestUser) and review VPN usage.
      1. Remove SRTL vpn access if not registered for other courses.
      2. Demand Yubikey back if not registered for other courses or for SRE access.
  5. Notify srtl@popdata.bc.ca after end of course for IT staff to do clean up folders for former students.

Course Calendar

  • NOTE: script fix-srtl-perms.sh must be edited each term by IT staff to adjust active courses and instructors,
    else access to files by students may fail. After editing, run manually to check for errors.
SRTL Course INSTRUCTOR START SRTL START END STATUS
PHDA-01 Working with Admin. Data Kim Nuernberger 2019-09-09 2019-09-25 2019-12-10 Vetting ???
PHDA-02 Epidemiological Stats Allison Scott 2020-01-14 2020-01-21 2020-04-14 Last SRTL access closed April 14, 2019, vetting requested by May 7, 2019
PHDA-03 Pop. Health and GIS Anders Erickson 2019-09-09 2019-09-16 2019-12-10 No MoH data; Vetting ???
PHDA-04 Spatial Epidemiology Anders Erickson 2020-01-14 2020-01-21 2020-04-14 Last SRTL access closed Dec 10, 2018, International access; No MoH data; Next delivery January 14, 2020
PHDA-05 Longitudinal Analysis Laura Holder 2020-05-?? 2020-05-?? 2020-07-?? Internat'l access; No MoH data; Next Delivery, May 2019; SRTL access opens May 13, 2019
(PHDA-06 Health Srv. Prog. Mon. & Eval) Marla Steinberg 2020-01-14 N/A 2020-04-07 Does not use PopData servers
SPPH-531 Kim McGrail & Dimitra Panagiotoglou 2019-09 2019-09 2019-?? Active
SPPH-537 Patti Janssen 2019-10 2017-10-18 2019-12-?? Active

PHDA Courses "The Professional Specialization Certificate in Population Health Data Analysis"

See https://www.uvcs.uvic.ca/population/

PHDA01 on srtl 2016-05-09 "Working with Administrative Data" (Old 11-s07)

  • Next Delivery: Sept 9 to Dec 10, 2019

Instructor: Kim Nuernberger SRTL access: Sept 18th, 2019 SRTL Course work starts: Sept 25th, 2019

Past Delivery - Sept 2018

  • Past Moodle course started 2018-09-11 to 2018-12-02, SRTL access Sept 24,2018, SRTL course work Sept 24, 2018
  • Instructor: Kim Nuernberger
  • Past Moodle course started 2016-09-09 to 2016-12-04, SRTL access Sept 18, 2016 SRTL coursework Sept 25, 2016
  • Instructor: Kirstin Atwood for Sept 2014, Sept 2015 and May 2016 Course Deliveries
  • New Instructor: Kim Nuernberger for Sept 2016, Sept 2017
  • Original home of infamous "MoH training dataset"
  • Originally offered as SRE "project" 11-s07

Past Instructors: Kim McGrail, Mieke Koehoorn

PHDA02 on srtl 2016-01-11 "Epidemiological Statistics"

  • Next Delivery - January 14 to April 14, 2020
  • Instructor -Allison Scott - Instructor Bio on this page: https://www.popdata.bc.ca/etu/PHDA/courses/02
  • Last Delivery - Jan 14 to April 7, 2019 - Allison Scott Instructor
  • Past Delivery - May 7 to July 29, 2018 - Allison Scott Instructor
  • Past Delivery - Jan 9 to April 2, 2017
  • Past Moodle course delivery - Jan 11 to April 3, 2016
  • Originally on RTL ending 2012-04-16. Prev instructor Larry Frisch
  • 2013-09-09 Transfer files from other courses:
    • From phda-05 copy "MoH training dataset": R:/DATA/2012-07-05/docs/ [5M] R:/DATA/2012-07-05/hospital/ [55M] R:/DATA/2012-07-05/registry/ [47M] R:/DATA/2012-07-30/msp/ [117M]
    • from rtl/Courses/PHDA02/ move:
      • "SAS tutorials" [957M] to Course_files/ (Esp. sub-folders: demographics/ hospital/ msp/ )
      • DATA/JCUSH [47M] & DATA/SBP [<1M] to DATA/
      • Working to working
    • from srtl/phda-01 copy: course_files/datadict/ ; course_files/fmtlib/ ; course_files/sasprogs/ [5M] (including mk_data/ subfolders demographics/ hospital/ msp/)
      • Instructor for all past deliveries: Larry Frisch: Jan 2012, Jan 2013, May 2014, Jan 2015, Jan 2016, Jan 2017
        • SRTL access - week 2 of the course

PHDA03 on srtl 2016-05 "Population Health and Geographic Information Systems (GIS)"

  • Next Delivery - Sept 9 to Dec 10 2019
  • Instructor: Anders Erickson
  • No MoH training dataset, international SRTL access continues
  • Past Delivery - May 7 to July 29, 2018 - Instructor, Anders Erickson
  • Past Delivery - Jan 9 to April 2, 2017 - Instructor, Anders Erickson

Moodle course starts May 9 to July 31, 2016

  • Originally on RTL 2012-05
  • In 2015 Instructor Alejandro Cervantes (acervant-phda-03) ; First Delivery of SRTL International Access
  • In 2016 Instructor Alejandro Cervantes; Planning for SRTL International Access
  • In 2017 Instructor Anders Erickson - SRTL International Access continues
  • Not using MoH training data
  • TestUser-phda-03 created

SRTL access - week 2 of the course

PHDA04 on srtl 2016-09 "Spatial Epidemiology and Outbreak Detection"

  • Next Delivery - January 14 to April 14, 2020
  • Instructor: Anders Erickson
  • No MoH training data, international SRTL access continues
  • Last Delivery - Sept 11 to Dec 2, 2018
  • Past Delivery - Sept 12- Dec 4 2016

Instructor: Anders Erickson Planning for International SRTL access as we did for last intake of PHDA 03

  • First Delivery in May 2014

Instructor: Anders Erickson; first viewing phda-03. Use account "aerickson" email: anderse@uvic.ca

  • Not using MoH Training Data Set

Next Delivery planned for Sept 10 to Dec 2, 2018 - International Access continues Instructor: Anders Erickson SRTL access - week 2 of the course

PHDA05 on srtl 2017-01 "Longitudinal Analysis and Multi-level Modeling of Population Health Data"

Next Delivery: May to July 2019 Instructor:Laura Holder No MoH training dataset, international access continues

Past Delivery: May 8 to July 30th, 2017 Instructor: Shayesteh Jahanfar Removed MoH training dataset and provided International SRTL access. Course uses only NPHS dataset

  • Last Moodle course started 2015-09-14 to 2015-12-06; SRTL access ~ Oct 1, 2016. Instructor Shayesteh Jahanfar; SJahanfar-phda-05
  • Originally on RTL 2013-03; Instructor from 2013 to 2014 Rubab Arim
  • 2014-08-13: moved NPHS (National Population Household Survey / Canada) data from rtl/Courses/PHDA05/DATA/ to DATA/NPHS/ {Bootstrp/ Data/ Doc/ Layout/ LisezMoi.pdf ReadMe.pdf }
  • 2014-08-13: changed DATA folder to match phda-01 (single folder 2013-12-10 with corrections integrated replaces 2012-07-05 & 2012-07-30)

SRTL access - week 2 of the course

PHDA06 NO SRTL or RTL COMPONENT 2016-01 Health Services Program Monitoring and Evaluation

Next Delivery: January 14 to April 14,2020 Instructor: Marla Steinberg

Last Delivery: Jan 14 to April 7, 2019 Past Delivery: Sept 11 to Dec 3, 2017 Instructor: Marla Steinberg No SRTL or RTL component, No privacy testing

Past Delivery: January 11 to April 3 2016 Instructor: Marla Steinberg

No SRTL or RTL component, No privacy testing in this course

Other courses

SPPH537 on srtl 2013-09 "Perinatal Epidemiology"

SPPH581D on srtl 2013-09 "Health Care Systems Analysis"

Instructor Kim McGrail getting Yubikey from Kaitlyn ~Sep.3

Course planning

phda03 starting around 2012-05-07

Hardware Planning: SRE / RTL / SRTL

  • Host Machines (vmware) can be on any vmware server
  • separate VPN networks for each set of outside clients (and their RDP/SMB servers)
  • separate networks for each set of virtual machines
  • set up DNS aliases for: sreFiles/srtlFiles/rtlFiles => Fraser
  • Firewall rules
  • They all use same VPN server, but different group
SRE
  • For projects
  • Transfers: allowed (with restrictions); SMB://SREfiles.popdata.bc.ca
  • R: \\fraser\sre)
  • VPN group: SRE (SecurID authentication)
sRTL
  • For courses with sensitive data and/or restricted (ex: paid) access.
  • Transfers: none
  • R: \\sRTLfiles\srtl) , currently server Drake
  • VPN group: srtl (SecurID)
RTL
  • For courses with no sensitive data
  • Transfers: allowed (with same restrictions as SRE); SMB://RTLfiles.popdata.bc.ca
  • R: \\fraser\rtl)
  • VPN group: course (LDAP password)

Creating a course on RTL

Create group course-phda02

In PDS under tab Groups->SRE Groups; at bottom [Add Group]; select initial members. On Fraser under /data/sre/Courses/ create PHDA02 in the group, owned by speterson, blocked from non-members

Create student user manually, assign to course

In PDS,

  • Create user: Tab Users -> Tasks -> [Create User]: FirstName, LastName, UserName, eMail, Group=External, Password
  • Activate User: Tab SRE -> User Logins -> Search and select user -> edit Active=yes; Activated for SRE Courses=yes; password reset=no
    • Alternate: SRE->Users->edit: Active=yes, course user=yes (automatically get in group rtl-users, vpn rtl)
      • Note that disabling "course user" does not disable VPN access, which is done in tab [SRE] -> sub-tab [Tasks] -> [VPN] -> select user from pull-down list (alphabetic by first name), group=none . This places user in VPN "employee" group "pd-no-access", which has a policy of 0 maximum simultaneous logins.
    • then add to course group course-phda02
  • Tell user to change password, check later for "password reset".
  • Note that SRTL does not work well with PDS. Changes done manually to join srtl-users on Active Directory on Gilbert.

Creating a course on SRTL

  • In PDS open tab "groups" , and click [add group] ; specify the name (all lower-case, one hyphen (-), and location "SRE Projects". Don't add members now, they will be added later when you create secondary accounts.
  • Next create an instructors group: the same name as above plus suffix "-instr"; location "SRE Sub-groups", .
  • Add instructor to new course group and instructors group.
  • Run create_srtl_course & create_srtl_user_dirs COURSE_NAME
  • Edit in /usr/local/sbin/ create_srtl_user_dirs & fix-srtl-perms.sh
  • remind instructor to refresh passphrase under my.popdata

Import/Export restricted for SRTL

  • SRTL courses have no "self-service" import or export.
  • For import:
    1. Files to be imported should be attached to an email to srtl@podpata.bc.ca , along with a description of their individual contents and origin. The OTRS ticket will thus contain a permanent record of the files and description.
    2. PopData IT staff will extract the files, examine them, and if they don't contain data
    3. transfer to SRTL file-server Drake under the instructor's personal folder, and set permissions to owner=instructor, readable by course group.
    4. All data should first have a copy put in the Repository, see below Courses#Adding_data_to_an_SRTL_course
  • For export
    1. Files to be exported should be individually listed and described in an email to srtl@popdata.bc.ca.
    2. PopData IT staff will extract the files, examine them, make a temporary copy on their desktop machine, and attach them to an email from the OTRS ticket system.

Folders and permissions for SRTL courses

E.g. for course phda-03, COURSE_GROUP is "phda-03", each USER matches *-phda-03 (two dashes, "*-${COURSE_GROUP}" ), INSTR is phystad-phda-03. INSTR_GROUP would be named "phda-03-instr" ( "${COURSE_GROUP}-instr" ).

SRTL courses have the following folder structure

  • The "R:" folder for all users (instructors and students) is the course folder, same as SRE project folder.
  • Subfolders are:
    • Personal folders for each student and instructor, named by course-specific account. Ex: R:\testuser1-phda-01 . Note that SRTL personal folders (unlike SRE and RTL) do not have a "TRANSFER" sub-folder.
    • R:\DATA\ : read-only copy of original protected data.
    • R:\course_files\ : readable by all, writeable by instructors.
    • R:\working\ : a sub-folder for each user, readable by all, for cooperation
    • R:\SUBMIT\ : a sub-folder for each user, writeable by user and instructors (but blocked from others), for grading/correcting. A script "fix-submit-permissions.bat" is provided for the instructors to make their changes readable by the student who owns each folder. A cron job every 15 minutes fixes permissions for all courses known to be active.
  • 2016-04 Request by Ann for phda-03 "working" subfolders to be writeable by all students, so they can comment on each other's files. Note that MS Office when re-writing a file does: create new file, delete old file, rename new version, which requires write access to directory. This requires manual intervention after each account creation.
Folders - technical details
  • Folders:
    • ${USER} (private for each user) owner=${USER} group=${COURSE_GROUP} ; permissions: files: u=rw,go= ; directories: u=rws,go=
    • working/${USER} (shared for group cooperation). owner=${INSTR}
    • SUBMIT/${USER} (individual subdirectories owned by ${USER} and writeable by group ${INSTR_GROUP})
      • The only way to give instructor access is via group permission, so all courses with a SUBMIT directory will need an INSTR_GROUP.
      • After instructor edits the document, or creates a new one, it belongs to instructor, and becomes inaccessible to student who owns folder but not this new file. File-ACL "default" option don't work with SAMBA, which explicitly sets permissions on new files. Script srtl/phda-03/SUBMIT/testuser1-phda-03/fix-submit-permissions.bat allows read access to all contents, while leaving each folder accessible only by student owner and instructors group. There is a copy in SRTL/.scripts , which should be copied to testuser1's SUBMIT folder.
        • R: ; cd \SUBMIT ; for /D %%D in (*) DO icacls %%D\* /grant Everyone:rx /T ; pause press Enter to close window.
    • DATA (protected data including personal records) Owned/readonly by root; readable by $GROUP
    • course_files (code, docs and probably data derived from DATA folder) owned by INSTR; group one of two options:
      • -either- read-only by ${COURSE_GROUP}
      • -or- writeable by ${INSTR_GROUP} *AND* read-only by others. Note that here, others can only be members of COURSE_GROUP, because parent directory is only accessible to COURSE_GROUP.
    • Permissions need occasional clean-up after uploads and/or moving files into course_files.
      • COURSE_GROUP="phda-03"; INSTR="phystad-phda-03"; INSTR_GROUP="phda-03-instr"
      • cd /data/srtl/phda-03
      • find course_files -type d -not \( -user $INSTR -a -group $INSTR_GROUP -a -perm -050 \) -ls -exec chown $INSTR:$INSTR_GROUP \{} \; -exec chmod g=rwxs,o=rx \{} \;
      • find course_files -type f -not \( -user $INSTR -a -group $INSTR_GROUP -a -perm -040 \) -ls -exec chown $INSTR:$INSTR_GROUP \{} \; -exec chmod g=rw,o=rx \{} \;
    • Extras (currently points to rtl/Courses/PTUT01 ; later may point to other shared resources)
    • Hidden System folders
      • .scripts ( -> /data/srtl/.scripts )
      • .login (record-keeping. -> /data/srtl/.login )
      • .libs (some software packages use extensions in sub-folders PYLIBS, RLIBS and/or SPSS_EXT. -> /data/sre/.libs )
  • Script on Drake "fix-srtl-perms.sh" enforces parts of the above scheme (i.e. "DATA", "course_files" and "SUBMIT", but not "working") every 15 minutes. *: Option "-n" (not-really, to see which files/directory are non-conforming) , arguments GROUP_NAME INSTRUCTOR_LOGIN.
  • Script "create_user_dirs" sets permissions for personal and working folders.

Folders and permissions for RTL courses

  • 2018-07-10 Special setup for student Jonathan Simkin (jsimkin) to save files under SUBMIT/jsimkin/ in courses "GIS_epi" and "Spatial_Epi_Tutorial"
    • with matching script fix-rtl-perms on Drake (run in crontab from fix-srtl-perms.sh).

Adding data to SRTL course

The procedure for adding data to an SRTL course is

  1. Collect information (name, source, fields, description, restrictions, size ...)
  2. Privacy Officer to approve
  3. Update Repository list "SRTL Dataset Repository.xlsx" in Alfresco/Education & Training/Training Datasets/00 Background Info, Timeline and Tracking Documents/
  4. Copy data in ready-to-use form to Fraser /sredata/Repository/srtl/{CourseNumber}/{CopyDate}
  5. Copy data to course DATA folder

Data Upload Record

  • RN added data to PHDA03 (2016-07-15)
* Received an additional data upload request from Alejandro Cervantes PHDS03 instructor.
* Collect information (name, source, fields, description, restrictions, size ...)
* Privacy Officer to be notified 
* Ann received data from Alejandro and encrypted with 7z
* Ann used secure.popdata.bc.ca site to upload files 
* Moved files to fraser:/sredata/Repository/srtl/phds-03/2016-07-15 [Repository moved to Drake 2016-08-30]
* Privacy Officer to review and approve files on SRE (by using the test project account)
* Ryoko '''did not update''' Repository list "SRTL Dataset Repository.xlsx" in Alfresco/Education & Training/Training Datasets/00 Background Info, Timeline and Tracking Documents/ 
* Copy data to requested folder drake:/data/srtl/phda-03/course_files/PHDA03/Final_Project_Data/Census/Census_2011 
* Permission set
  • DN uploaded data to PHDA04 (2016-08-31)
  • RN uploaded data to PHDA03 (2016-11-07)
* Received an additional data upload request from Anders Erickson PHDS04 and PHDS03 instructor via Ann.
* Files were arrived as an email attachments
* DN collected the data information (name, source, description, etc) and updated Alfresco/Education & Training/Training Datasets/00 Background Info, Timeline and Tracking Documents/SRTL Dataset Repository_2016-08-30_DL.xlsx
* DL copied files to drake:/sredata/Repository/srtl/MHMCatlas
* DL moved files to drake:/data/srtl/phda-04/MCMCatlas 
* RN moved files to drake:/data/srtl/phda-03/MCMCatlas 
* Permission set
  • RN uploaded data to RTL course (2016-10-07)
* Received a sample NLSY training data from Ann and SEM workshop instructor Piotr Wilk. 
* Ann provided 1) authorization letter from the data provider to be used at the workshop 2) public website link where you have access and to download.
* Privacy Officer to be notified. 
* Ann was onside (at Ryoko's desk) to discuss so I skipped the "secure upload" process. 
* RN archived them to drake:/sredata/Repository/rtl/SEM_Tutorial (<-Denis advised this part may not be necessary) 
* RN copied them to the live file server - fraser:/data/rtl/Courses/SEM_Tutorial (chmod 774 & chown agreenwood:rtl-instr <- Ann's instruction)
  • RN uploaded copied data from PHDA03 to PHDA04 (2016-11-16 & 2106-11-18)
* Copied drake:/sredata/srtl/phda-03/course_files/PHDA03/lab5/Data/HRs -> /sredata/srtl/phda-04/DATA/HRs (refer otrs ticket for the file description)
* Copied drake:/sredata/srtl/phda-03/course_files/PHDA03/Lab1/Data/Van_NDVI -> /sredata/srtl/phda-04/course_files/ExtraData/Van_NDVI (refer otrs ticket for the file description)
* It's not a new file upload request so nothing further needed to be done. 
  • RN uploaded and replaced a file on to PHDA02 (2017-01-16)
* Replaced drake:/sredata/srtl/phda-2/course files/Activities/PDFs of assignments/PHDA02_Module 1 Exercise_ungraded
  • RN uploaded new dummy SAS datasets to RTL (2017-04-07)
*fraser:/sredata/home/rtl/Courses/Data Management for Cleaning and Analysis/
*  Description of Dummy data sets_Data Management and Cleaning for Analysis.docx
*  raw_adt.sas7bdat
*  raw_casemix.sas7bdat
*These dummy datasets have been created by Brandon Wagar, the instructor who is currently delivering the webinar-based course "Data Management and Cleaning for Analysis".

PDS for student account creation

PDS for RTL

Here are the controls for accessing various pieces of RTL, all modifiable in PDS.

Take for instance user testrtl1

  1. VPN access to group "rtl" allows logging in to VPN, which then provides the ability to attempt a Remote Desktop login to some RTL machine. A user can be simultaneously member of multiple VPN groups.
  2. "RTL user: yes" allows remote desktop login to RTL machines. Similarly for "SRE user" and "SRTL user". Note that SRE users are normally set up by editing their main username, clicking "Create Project Login".
  3. group membership in "course-phda02" allows access to folders for course phda02. A user can be member of multiple groups.

Here is how one uses PDS to change membership in permission groups like "course-phda02":

  • Click on tab "Groups", then sub-tab "SRE Groups". In this case "SRE" means also "SRE-like" and includes RTL and SRTL.
    You have the option of focusing the list by specifying for example "course" in the search box and pressing Enter.
  • Click on "Edit" for the appropriate group. You should see 2 columns, titled "Available Members" and "Chosen Members". Complain if you see only one column [add username to "staff" group at https://pds.popdata.bc.ca/admin/auth/user/ ]
  • In the box "Filter" enter a substring of either the first name, last name, or username. Ex: "wood".
  • To add, click on the appropriate user entry, then on the right arrow.
  • To remove a member from the group, use the left arrow.
  • When all changes are done, click update.

Any navigation away trom the "Edit group" page other than via "Update key" cancels the changes.

PDS for SRTL

see Systems/Account_Management#Process_for_creating_SRTL_course-specific_accounts

Expiry / disabling of course users

Disable a single course user

  • For Primary account,
    • Unless they are registered in another SRTL course, remove VPN group "srtl" using button [Change VPN]. This prevents them from trying to connect to SRTL. To remove the last VPN entry for a user or add a second one you may need to hold CTRL or CMD key.
    • Arrange for return of Yubikey (and refund deposit if any). Once Yubikey is back delete it in PDS
    • The primary user can keep VPN "rtl" and checkbox "activated for RTL" if there's any chance they may want to take a free course.
  • and for secondary (Course) account
    • "Activated for SRTL" = No . This disables them from Remote Desktop sessions on SRTL machines.
    • Before the next course starts all students have to be removed from the course group, but it's more convenient in a batch later.
    In the case where the instructor later uses PDS to send an email to all group members, it would be better to have removed dropouts. That function does not use BCC and is probably more suited for emergency messages.


Disable multiple course users

See also Systems/Account_Management

All members (one exception is instructors) can normally be disabled as soon as the course is over . Before removing from group, you may want a list of current members. You can always recover the list later from the PDS Log for that group. Note that archiving course users' files can be done after disabling users, because disabling does not delete files.

In PDS:

  • Display group (ex: phda-02)
  • For each student, CTRL-SHIFT_CLICK on each username (open in new tab)
    • Near bottom, click on "Edit", and uncheck "Activated for SRTL". Of course there should be no VPN or YubiKey on a secondary account. Click "Update"...
    • Near bottom, click on "Primary User"
    • Unless they are registered in another SRTL course, remove VPN group "srtl". Consider leaving them with VPN group "rtl" and "activated for RTL".
    • Unless they have a remaining VPN access (e.g. "sre"), or will be using the yubikey soon for another purpose, arrange to recover the key and remove the YubiKey number from their primary account.

Archive and delete course user directories

srtl-archive-student (usr/local/bin/ & /usr/local/src/SRE/) takes 3 arguments:

  • -n (not-really) optionally just displays what action would be taken
  • COURSE
  • STUDENT (one or more)
  • Ex: /usr/local/bin/srtl-archive-students -n phda-01 TestUser-phda-01 jim-phda-01
    For each student specified, create an archive (/sredata/Archived/srtl/COURSE/STUDENT-YYMMDD.tgz) of files under personal, working, and SUBMIT folders (skipping empty folders and TRANSFER). Then display finished archive, and ask for confirmation before deleting. Archive name includes a date stamp, and if run twice the same day for the same student a counter.
  • Note: the student accounts should be removed from the course group before getting deleted, else on the next hour the folders will be re-created.
  • After archiving is done, look up the starting date of course, create a sub-directory (ex: 2016-09) and move all the new archives to it.

lists of course users and their activity

  1. Moodle activity report (could get from Ann)
  2. All accounts are listed monthly on Sullivan in /home/www/pds/logs/SRE-lists/
  3. SRE windows login on Drake:/data/srtl/.login/logins.txt
  4. in PDS under "VM Logins"
  5. Unix command "groupinfo" lists all members of group and their most recent Windows login.

Disabling IMPORT/EXPORT by students

Policy on Import/Export

  • 2011-dec-15 Denis, Jim & Nancy decided that RTL users will not have access to Import/Export functions (mostly for basic security). Sandra altered “PHDA_Orientation_to_RTL.docx” . This policy was never enforced, and the drop folder was never set up. Note that instructors could have set it up if they new how.
  • 2012-jan-13 Larry Frisch convinces Sandra to provide import/export. "Subject: RE: PHDA02: Question about assignments..."
  • 2012-jan-15 Sandra asks Jim to review instructions for import/export, he says OK.
  • 2012-mar-05 Sandra provides Denis with emails on subject.

Alternative to import/export: drop folder "SUBMIT"

Folder SUBMIT within course has a subfolder for each student, writeable by instructors group; see also script fix-submit-perms.bat for instructor to allow students to read instructor responses.

Delivering marked assignments after vetting

After course ends, students can request that their assignments be returned to them after marking. See Vetting_Training_Data_Set_Assignments * Students must clearly mark in their SUBMIT folder which files they want returned. Strict conditions apply to contents.

  • Some DSU staff gets added to instructors group for the course , thus getting full access to SUBMIT sub-folders.
    • IT staff create SUBMIT/VETTED, owned by vetter, instructors group.
    • for each student wanting files, data is cleaned out of assignment files, and clean copy is put under folder SUBMIT/VETTED
    • (Ryoko can't find these archive vetted files. Need to clarify with Denis. 2016-07)IT staff make a separate zip archive for each student vetted folder to permanent storage under /sredata/Archived/srtl/
      • Ex: cd /data/srtl/phda-01/SUBMIT/VETTED && sh /usr/local/src/SRE/srtl-zip-vetted.sh /sredata/Archived/srtl/phda-01/VETTED-1401 *
      • Ex of archive filename: bchou-phda-01-140115-NQ18HEjj.zip (username-timestamp-randomstring.zip)
    • Give student vetted zip files to Ann via email attachment. Ann will distribute vetted files to each students.

Import/Export Components on R: drive folders

  • Files accessible from ouside on Fraser:/data/sre/share/
    • incoming/$U/
    • outgoing/$U/
  • Accessible from SRE on Fraser:/data/sre/
    • sre/EXPORT_FROM_SRE/$U/ (also ../.trigger/ )
    • sre/IMPORT_FROM_SRE/$U/ (also ../.trigger/ )

Projects have an added layer of folders and symlinks.

Ex: /home/$U (accessed as smb://$U@fraser.popdata.bc.ca/$U) contains links

  • EXPORT_FROM_SRE -> /data/share/outgoing/$U
  • IMPORT_TO_SRE -> /data/share/incoming/$U

and from project folder /data/sre/$PROJ folder $U/TRANSFER/ contains links:

  • EXPORT_FROM_SRE -> /data/sre/EXPORT_FROM_SRE/akaltaye-11-s07
  • EXPORT-IT.lnk -> /data/sre/.scripts/EXPORT-IT.lnk
  • IMPORT-IT.lnk -> /data/sre/.scripts/IMPORT-IT.lnk
  • IMPORT_TO_SRE -> /data/sre/IMPORT_TO_SRE/akaltaye-11-s07

Delete / do no create user-specific folders and symlinks involved in transfers

If a user has not had transfer-related folders and symlinks created, they cannot run yellowfolders and inisiate transsfer in<->out

Review re-creation of folders by cron jobs on Fraser:

  • create_homes : triggered by absence of /data/share/incoming/$U or /home/$U ! So we would have to create unusable folder, for example owned by root.
  • create_project : triggered by absence of TRANSFER folder. We don't plan restrict project usernames from transferring.

Suppress SAMBA login to Fraser import/export based on VPN IP range

In /etc/samba/smb.conf change for sections [outgoing] and [incoming]

  • from: hosts deny = 10.80.50.128/25 ("pd-file" - SRE machines)
  • add: hosts deny = 10.80.18.0/24 ("course-vpn-pool" - VPN addresses for pd-course-policy)

Consider changing this blacklist to whitelist: include IP ranges for

  • sre-vpn-pool 10.80.15.0/24
  • sretest-vpn-pool 10.80.16.0/24
  • remote-pool 10.80.17.0/24
  • admin-pool 10.80.12.10 - ...30
  • chspr-client 10.30.10
  • pd-client 10.80.10 (incl. denis desktop NMISTRY1)
  • denis-macbook 10.40.10.0/24

Individual issues

WGan ASHG101 Wenqi Gan <wenqi.gan@gmail.com> username not created [...]

Oct-31 complained: can't login to portal ... "Question about Tutorial Data". "WGan" Not in PDS. Trace cartier:/etc/mail/aliases "uvic: | /usr/local/bin/external.py" -> /home/www/pds/general/external.py

Status: [1] uVic has started again sending emails to uvic@popdata ; [2] Jim has fixed typo in PDS that caused crash, so auto registration is working again ; [3] ToDo: fix PDS to edit groups; [4] ToDo: creat account WGan

Password change at uVic does not change password on related SRE course account

Nov-01 email from Emily Schudel <eschudel@uvic.ca> via Ann Greenwood "FW: [RT #256921] Benedito Chou | ASHG101 | Access to the SRE" quotes Manesh "you changed your id/password ... not be able to use this new id/password for access to your SRE".