Skip to content
Morrison-LabPublic

About

Website for searching WWU courses

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

24 Commits

Folders and files

Repository files navigation

WWU Course Explorer

A search site for Western Washington University courses, modeled on ExploreCourses. Search by code, title, keyword, instructor or department, filter by subject, term, level, attribute (GUR, writing proficiency, ...) and delivery mode, and see every section of a course in the loaded terms, with its instructor, meeting times and enrollment.

Data sources

Source What it gives How
Browse Classes (Ellucian Banner 9) Sections for any term back to Fall 2002: instructor, times, rooms, enrollment, delivery, attributes; the course catalog with college, department, descriptions, prerequisites and corequisites Public JSON endpoints, no login
University Catalog (Acalog) Official course pages Linked from each course; the site blocks scripted requests, so it is not scraped
data/syllabi.csv Syllabus and course-material links Curated by hand
Department websites, from the department directory Instructors' profile pages scraper/profiles.py reads each site's people pages for names and links, weekly

WWU retired its TimeTable/ClassFinder on 2025-10-07; Browse Classes replaced it.

By default the scraper starts from Winter fifteen calendar years before this one (Winter 2011 in 2026), set by HISTORY_YEARS in scraper/scrape.py. Terms already downloaded stay in the cache, so the history grows from there rather than rolling forward. Earlier terms are one flag away (--since 200510, or --all-terms for everything since 2002). Past terms are downloaded once and then kept; the newest six are refreshed on every run.

Privacy

Banner's public search has no student data beyond section enrollment counts. The scraper keeps only each section's primary instructor (the instructor of record), by name. It drops secondary instructors, who are often graduate teaching assistants, and instructor e-mail addresses and Banner IDs. A graduate student who is the instructor of record still appears, as on WWU's own Browse Classes.

Run it locally

pip install requests
python3 scraper/scrape.py      # sections, catalog, descriptions, requisites -> data/raw/
python3 scraper/profiles.py    # instructors' profile pages -> data/raw/profiles.json (optional)
python3 scraper/build.py       # -> site/data/courses.json and graph.json, site/course/, site/instructor/
python3 -m http.server -d site 8000

Course descriptions are fetched one at a time, with a pause between requests, and cached in data/raw/descriptions.json, which is committed, so a run fetches only courses missing from it. --max-descriptions N caps that for a quick test. Prerequisites and corequisites work the same way, cached in the committed data/raw/requisites.json (--max-requisites N); a course in the catalog is looked up in the newest term, and one that has left it in the last term it was offered. Banner lists only the requisites it enforces, and they can change from year to year, so rerun with --refresh-requisites and commit the file now and then. A full download takes a few hours, so the daily workflow looks up at most 1,500 new courses a run (--max-requisites 1500) and the seed is refreshed locally.

Publishing

.github/workflows/course-explorer.yml re-scrapes daily, rebuilds, and deploys site/ to GitHub Pages. It needs Settings > Pages > Source: GitHub Actions turned on once. Run it by hand from the Actions tab, optionally with a since term code to load more history.

Downloaded terms carry over between runs in the Actions cache, not in git. If that cache is evicted (GitHub drops caches unused for 7 days), the next run downloads the fifteen years again, and terms older than that loaded by hand drop off the site until a run with that since is repeated. If a term's new download has under half the sections of the cached copy, the scraper keeps the old copy, finishes the other terms, and then fails the run, so a truncated schedule is never published. If the drop is real, run the workflow by hand with allow_shrink checked.

Adding syllabi

Add a row to data/syllabi.csv:

course,term,instructor,url,label
BIOL 204,202640,"Doe, Jane",https://example.edu/biol204-syllabus.pdf,Syllabus

One row per link:

  • course: the course as Banner writes it, subject and number (e.g. EPI 204).
  • term (optional): the Banner term code (e.g. 202640 for Fall 2026); the build stops on anything else.
  • instructor (optional): whose syllabus it is.
  • url: an http:// or https:// link; the build stops on anything else.
  • label (optional): the link text, Syllabus if empty.

Keep the file plain CSV, with no comment lines, so GitHub shows it as a table. Link only public pages, and nothing that names a student.

WWU has no public syllabus archive, so the list is built by hand. The first rows come from faculty course sites that are public and list no students, and from the sample syllabi WesternOnline publishes for its self-paced courses. Instructors can make a Canvas syllabus page public, but few do and nothing lists them. Each course's card and page also link to the WWU Bookstore's textbook lookup and to a Suggest a syllabus link issue form, filled in with the course; a suggestion becomes a row here once someone checks the page.

Layout

  • scraper/scrape.py: downloads from Banner into data/raw/
  • scraper/build.py: merges sections, catalog, descriptions, requisites and syllabi into one index
  • scraper/requisites.py: turns Banner's prerequisite and corequisite rows into course links, and finds the courses that require each course
  • scraper/profiles.py: finds instructors' profile pages on the department websites, as name and link pairs in the committed seed data/raw/profiles.json; build.py links an instructor page to the profile whose name matches
  • scraper/crosslist.py: links the codes one course is listed under, cross-listed (ENRG 482 and ESCI 482) or taught at two levels (GEOL 497T and 597T), from Banner's cross-list groups and sections that share an instructor and meeting times; search shows each such course once, and its card and page say where else it is listed
  • scraper/pages.py: writes a static page per course (its prerequisites and corequisites, the courses that require it, each as a list, a foldable tree and a graph, every section it has had in the listed terms, and who taught it) and per instructor (every section they taught in those terms, in one table that site/sections.js sorts by any column and can combine over terms and/or courses, and a link to their WWU profile when profiles.py found one), plus an instructor list; instructors are matched by the name Banner shows; it also writes site/data/graph.json, each course's title, page and the courses on either side of it, for the graph
  • site/: static page (index.html, app.js, style.css) searching the index in the browser with MiniSearch; multiselect.js, the checkbox dropdowns for the filters (picks within one filter match any of them, Attribute can be switched to match all, and the filters combine); theme.js, the lab sites' Light / Off-white / Parchment / Dark theme menu (the choice is shared with them under mln-theme); and reqs.js, on course pages, which adds Unfold all and Fold all to the requisite trees and draws the requisite graph with Cytoscape.js and its dagre layout, loaded from jsDelivr when the graph is first opened

About

Website for searching WWU courses

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages