{"id":1766267187227,"date":"2026-02-09T19:34:28","date_gmt":"2026-02-10T01:34:28","guid":{"rendered":"https:\/\/artsincubator.ca\/borupscorners\/?p=1766267187227"},"modified":"2026-03-10T10:18:40","modified_gmt":"2026-03-10T15:18:40","slug":"scrape-it","status":"publish","type":"post","link":"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/","title":{"rendered":"Scrape it!"},"content":{"rendered":"\n

How to Digest 1,000 Government Comments Before Breakfast<\/h2>\n\n\n\n

How do you make sense of hundreds of impact assessment comments without losing your mind? The old way involves clicking a link, reading a page, copying text into a spreadsheet, hitting the back button, and repeating that process until your eyes cross. It\u2019s a slow, manual grind that makes true civic engagement exhausting.<\/p>\n\n\n\n

So, we decided to cheat. Well, not cheat\u2014automate<\/em>.<\/p>\n\n\n\n

In about five minutes, we built a custom web scraping tool specifically designed to tackle the massive amount of public data surrounding the NWMO Deep Geological Repository (DGR) Impact Assessment<\/a>. Instead of navigating the Impact Assessment Agency of Canada (IAAC) website<\/a> manually, we built a robot to do the heavy lifting for us.<\/p>\n\n\n\n

The 5-Minute Build<\/strong>
We spun up a lightweight web application that acts as a tireless research assistant. The concept is simple: the government registry stores comments with sequential IDs in their URLs. Our tool takes a starting ID (like 65040) and an ending ID (like 64000) and loops through them automatically.<\/p>\n\n\n\n

It visits each page in the background, bypasses standard browser restrictions (thanks to a nifty CORS proxy), and reads the HTML source code instantly. It looks for specific patterns\u2014bold labels like “Submitted by,” “Reference number,” and “Phase”\u2014and extracts the text next to them. It even hunts down attachment links for PDFs. If a page is empty or an ID is missing, the tool simply notes it and moves on to the next one, faster than a human could ever move a mouse.<\/p>\n\n\n\n

Data Democracy<\/strong>
The result? A clean, structured table populated with hundreds of records in real-time, all exportable to a CSV file with a single click.<\/p>\n\n\n\n

This changes the game for analysis. Instead of treating each comment as an isolated web page, we now have a dataset. We can sort by date to see when engagement spiked. We can filter by “Submitted by” to see which organizations are most active. We can feed the raw text into an LLM to summarize sentiments or extract key themes across thousands of submissions.<\/p>\n\n\n\n

Why Wait?<\/strong>
Open data shouldn’t just mean “technically available on the internet.” It should mean accessible, usable, and analyzable. Often, valuable public information is hidden behind clunky user interfaces and paginated lists that deter the average citizen.<\/p>\n\n\n\n

Technology bridges that gap. Why wait weeks for an Access to Information request or spend days manual-entrying data when you can build a scraper in the time it takes to brew a coffee? With a little bit of code (and some AI assistance to write it), we can turn an overwhelming wall of bureaucracy into a transparent, manageable spreadsheet.<\/p>\n\n\n\n

That\u2019s how you power modern democracy\u2014one scraped record at a time.<\/p>\n","protected":false},"excerpt":{"rendered":"

Open data shouldn’t just mean “technically available on the internet.”<\/p>\n","protected":false},"author":1,"featured_media":1766267187228,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20,333],"tags":[199,332,232],"class_list":["post-1766267187227","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-photos","category-workshops","tag-northwestern-ontario","tag-ontario-arts","tag-sdg11"],"yoast_head":"\nScrape it! - Art Borups Corners<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Scrape it! - Art Borups Corners\" \/>\n<meta property=\"og:description\" content=\"Open data shouldn't just mean "technically available on the internet."\" \/>\n<meta property=\"og:url\" content=\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/\" \/>\n<meta property=\"og:site_name\" content=\"Art Borups Corners\" \/>\n<meta property=\"article:published_time\" content=\"2026-02-10T01:34:28+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-03-10T15:18:40+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2026\/02\/scraping-nuclear-data-iaac-impact-assessment.jpeg\" \/>\n\t<meta property=\"og:image:width\" content=\"1867\" \/>\n\t<meta property=\"og:image:height\" content=\"1050\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Art Borups Corners\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Art Borups Corners\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/\"},\"author\":{\"name\":\"Art Borups Corners\",\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/#\/schema\/person\/4b2906b824b21699f2a765b383e2c8db\"},\"headline\":\"Scrape it!\",\"datePublished\":\"2026-02-10T01:34:28+00:00\",\"dateModified\":\"2026-03-10T15:18:40+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/\"},\"wordCount\":457,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/#organization\"},\"image\":{\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2026\/02\/scraping-nuclear-data-iaac-impact-assessment.jpeg\",\"keywords\":[\"Northwestern Ontario\",\"Ontario Arts\",\"SDG 11\"],\"articleSection\":[\"Photos and Short Stories\",\"Workshops\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/\",\"url\":\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/\",\"name\":\"Scrape it! - Art Borups Corners\",\"isPartOf\":{\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2026\/02\/scraping-nuclear-data-iaac-impact-assessment.jpeg\",\"datePublished\":\"2026-02-10T01:34:28+00:00\",\"dateModified\":\"2026-03-10T15:18:40+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#primaryimage\",\"url\":\"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2026\/02\/scraping-nuclear-data-iaac-impact-assessment.jpeg\",\"contentUrl\":\"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2026\/02\/scraping-nuclear-data-iaac-impact-assessment.jpeg\",\"width\":1867,\"height\":1050,\"caption\":\"Screenshot\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/artsincubator.ca\/borupscorners\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Scrape it!\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/#website\",\"url\":\"https:\/\/artsincubator.ca\/borupscorners\/\",\"name\":\"Art Borups Corners\",\"description\":\"Melgund Township, Northwestern Ontario\",\"publisher\":{\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/artsincubator.ca\/borupscorners\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/#organization\",\"name\":\"Art Borups Corners\",\"url\":\"https:\/\/artsincubator.ca\/borupscorners\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2025\/07\/art-borups-corners-melgund-township-northwestern-ontario-northern-arts-program.jpeg\",\"contentUrl\":\"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2025\/07\/art-borups-corners-melgund-township-northwestern-ontario-northern-arts-program.jpeg\",\"width\":221,\"height\":221,\"caption\":\"Art Borups Corners\"},\"image\":{\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/#\/schema\/person\/4b2906b824b21699f2a765b383e2c8db\",\"name\":\"Art Borups Corners\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/artsincubator.ca\/borupscorners\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2025\/07\/art-borups-corners-melgund-township-northwestern-ontario-northern-arts-program-1-150x150.jpeg\",\"contentUrl\":\"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2025\/07\/art-borups-corners-melgund-township-northwestern-ontario-northern-arts-program-1-150x150.jpeg\",\"caption\":\"Art Borups Corners\"},\"description\":\"Art Borup\u2019s Corners is a northern arts incubator based in Melgund Township, Northwestern Ontario, where community-led creativity, land-based practice, and digital innovation come together. Rooted in the cultural rhythms of the boreal forest and shaped by years of grassroots organizing across Ontario, Manitoba, Nunavut, and Minnesota, Borup\u2019s Corners supports artists, youth, and community members through participatory storytelling, climate-focused projects, and creative entrepreneurship. From wild blueberry walks to immersive exhibitions and applied AI research, our seasonal programs and artist residencies foster connection, skill-building, and self-determined expression\u2014all grounded in place, culture, and care.\",\"sameAs\":[\"https:\/\/artsincubator.ca\/borupscorners\"],\"url\":\"https:\/\/artsincubator.ca\/borupscorners\/author\/lsbadmin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Scrape it! - Art Borups Corners","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/","og_locale":"en_US","og_type":"article","og_title":"Scrape it! - Art Borups Corners","og_description":"Open data shouldn't just mean \"technically available on the internet.\"","og_url":"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/","og_site_name":"Art Borups Corners","article_published_time":"2026-02-10T01:34:28+00:00","article_modified_time":"2026-03-10T15:18:40+00:00","og_image":[{"width":1867,"height":1050,"url":"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2026\/02\/scraping-nuclear-data-iaac-impact-assessment.jpeg","type":"image\/jpeg"}],"author":"Art Borups Corners","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Art Borups Corners","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#article","isPartOf":{"@id":"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/"},"author":{"name":"Art Borups Corners","@id":"https:\/\/artsincubator.ca\/borupscorners\/#\/schema\/person\/4b2906b824b21699f2a765b383e2c8db"},"headline":"Scrape it!","datePublished":"2026-02-10T01:34:28+00:00","dateModified":"2026-03-10T15:18:40+00:00","mainEntityOfPage":{"@id":"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/"},"wordCount":457,"commentCount":0,"publisher":{"@id":"https:\/\/artsincubator.ca\/borupscorners\/#organization"},"image":{"@id":"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#primaryimage"},"thumbnailUrl":"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2026\/02\/scraping-nuclear-data-iaac-impact-assessment.jpeg","keywords":["Northwestern Ontario","Ontario Arts","SDG 11"],"articleSection":["Photos and Short Stories","Workshops"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/","url":"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/","name":"Scrape it! - Art Borups Corners","isPartOf":{"@id":"https:\/\/artsincubator.ca\/borupscorners\/#website"},"primaryImageOfPage":{"@id":"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#primaryimage"},"image":{"@id":"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#primaryimage"},"thumbnailUrl":"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2026\/02\/scraping-nuclear-data-iaac-impact-assessment.jpeg","datePublished":"2026-02-10T01:34:28+00:00","dateModified":"2026-03-10T15:18:40+00:00","breadcrumb":{"@id":"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#primaryimage","url":"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2026\/02\/scraping-nuclear-data-iaac-impact-assessment.jpeg","contentUrl":"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2026\/02\/scraping-nuclear-data-iaac-impact-assessment.jpeg","width":1867,"height":1050,"caption":"Screenshot"},{"@type":"BreadcrumbList","@id":"https:\/\/artsincubator.ca\/borupscorners\/scrape-it\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/artsincubator.ca\/borupscorners\/"},{"@type":"ListItem","position":2,"name":"Scrape it!"}]},{"@type":"WebSite","@id":"https:\/\/artsincubator.ca\/borupscorners\/#website","url":"https:\/\/artsincubator.ca\/borupscorners\/","name":"Art Borups Corners","description":"Melgund Township, Northwestern Ontario","publisher":{"@id":"https:\/\/artsincubator.ca\/borupscorners\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/artsincubator.ca\/borupscorners\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/artsincubator.ca\/borupscorners\/#organization","name":"Art Borups Corners","url":"https:\/\/artsincubator.ca\/borupscorners\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/artsincubator.ca\/borupscorners\/#\/schema\/logo\/image\/","url":"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2025\/07\/art-borups-corners-melgund-township-northwestern-ontario-northern-arts-program.jpeg","contentUrl":"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2025\/07\/art-borups-corners-melgund-township-northwestern-ontario-northern-arts-program.jpeg","width":221,"height":221,"caption":"Art Borups Corners"},"image":{"@id":"https:\/\/artsincubator.ca\/borupscorners\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/artsincubator.ca\/borupscorners\/#\/schema\/person\/4b2906b824b21699f2a765b383e2c8db","name":"Art Borups Corners","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/artsincubator.ca\/borupscorners\/#\/schema\/person\/image\/","url":"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2025\/07\/art-borups-corners-melgund-township-northwestern-ontario-northern-arts-program-1-150x150.jpeg","contentUrl":"https:\/\/artsincubator.ca\/borupscorners\/wp-content\/uploads\/2025\/07\/art-borups-corners-melgund-township-northwestern-ontario-northern-arts-program-1-150x150.jpeg","caption":"Art Borups Corners"},"description":"Art Borup\u2019s Corners is a northern arts incubator based in Melgund Township, Northwestern Ontario, where community-led creativity, land-based practice, and digital innovation come together. Rooted in the cultural rhythms of the boreal forest and shaped by years of grassroots organizing across Ontario, Manitoba, Nunavut, and Minnesota, Borup\u2019s Corners supports artists, youth, and community members through participatory storytelling, climate-focused projects, and creative entrepreneurship. From wild blueberry walks to immersive exhibitions and applied AI research, our seasonal programs and artist residencies foster connection, skill-building, and self-determined expression\u2014all grounded in place, culture, and care.","sameAs":["https:\/\/artsincubator.ca\/borupscorners"],"url":"https:\/\/artsincubator.ca\/borupscorners\/author\/lsbadmin\/"}]}},"_links":{"self":[{"href":"https:\/\/artsincubator.ca\/borupscorners\/wp-json\/wp\/v2\/posts\/1766267187227","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/artsincubator.ca\/borupscorners\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/artsincubator.ca\/borupscorners\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/artsincubator.ca\/borupscorners\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/artsincubator.ca\/borupscorners\/wp-json\/wp\/v2\/comments?post=1766267187227"}],"version-history":[{"count":1,"href":"https:\/\/artsincubator.ca\/borupscorners\/wp-json\/wp\/v2\/posts\/1766267187227\/revisions"}],"predecessor-version":[{"id":1766267187229,"href":"https:\/\/artsincubator.ca\/borupscorners\/wp-json\/wp\/v2\/posts\/1766267187227\/revisions\/1766267187229"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/artsincubator.ca\/borupscorners\/wp-json\/wp\/v2\/media\/1766267187228"}],"wp:attachment":[{"href":"https:\/\/artsincubator.ca\/borupscorners\/wp-json\/wp\/v2\/media?parent=1766267187227"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/artsincubator.ca\/borupscorners\/wp-json\/wp\/v2\/categories?post=1766267187227"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/artsincubator.ca\/borupscorners\/wp-json\/wp\/v2\/tags?post=1766267187227"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}