initial pandoc export

README.md

@@ -4,54 +4,21 @@ This repository contains a script to pull semantic data out of Semantic Mediawik
 
 Built for [Security Vision](https://securityvision.io).
 
+## Document
+
+```
+pandoc EU\ Greens\ Report\ -\ Biometric\ Mass\ Surveillance\ v.3.docx --extract-media=images --template=template.html --section-divs -s -o output.html --metadata title="Biometric and Behavioural Mass Surveillance in EU Member States" -c mvp.css --toc --toc-depth=2
+```
 
 ## Installation
 
 ```bash
-pip install -r requirements.txt
 wget https://d3js.org/d3.v6.min.js
 ```
 
-## Update:
-
-```
-python wiki_relations.py
-```
-
 ## Data
 
-_Ask_ SMW with the following query:
-
 ```
-({{#ask: [[Category:Deployments||Institution]]  
-OR [[Category:Technologies]] [[Developed by (institutions)::+]]  // TODO: + should give a subquery <q></q> with all institutions in EU
-OR [[Category:Technologies]]  [[-Software Deployed::+]] // TODO: + should give a subquery <q></q> with all deployments in EU
- |?Category
- |?Geolocation
- |?City
- |?City.Has Coordinates=Has Coordinates
- |?City.Is in Country=City Country
- |?City Country.Has Coordinates=Country Coordinates
- |?Clients
- |?Managed by
- |?Used by
- |?Funded by
- |?Provided by // TODO: technology/product provided by
- |?Software Deployed
- |?Developped by (institutions)
- |format=broadtable
- |limit=500
- |offset=0
- |link=all
- |sort=
- |order=asc
- |headers=show
- |searchlabel=... further results
- |class=sortable wikitable smwtable
-}}
-
-TODO: see: https://www.semantic-mediawiki.org/wiki/Help:Querying_for_the_absence_of_a_property
-
 {{#ask: [[Category:Deployments||Institution]]  [[Is in Greens Report 2021::True]]
  |?Category
  |?Geolocation
@@ -81,10 +48,4 @@ TODO: see: https://www.semantic-mediawiki.org/wiki/Help:Querying_for_the_absence
  |searchlabel=... further results
  |class=sortable wikitable smwtable
 }}
-```
-
-Fetch using CURL:
-
-```bash
-curl 'https://www.securityvision.io/wiki/index.php?title=Special:Ask&#search' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Content-Type: application/x-www-form-urlencoded' -H 'Origin: https://www.securityvision.io' -H 'Connection: keep-alive' -H 'Referer: https://www.securityvision.io/wiki/index.php/Special:Ask' -H 'Upgrade-Insecure-Requests: 1' --data-raw 'title=Special%3AAsk&_action=submit&q=%5B%5BCategory%3ADeployments%7C%7CInstitution%5D%5D++&po=%0D%0A+%7C%3FCategory%0D%0A+%7C%3FGeolocation%0D%0A+%7C%3FCity%0D%0A+%7C%3FCity.Has+Coordinates%3DCity+Coordinates%0D%0A+%7C%3FCity.Is+in+Country%3DCity+Country%0D%0A+%7C%3FCity+Country.Has+Coordinates%3DCountry+Coordinates%0D%0A+%7C%3FClients%0D%0A+%7C%3FManaged+by%0D%0A+%7C%3FUsed+by%0D%0A+%7C%3FFunded+by%0D%0A+%7C%3FProvided+by%0D%0A+%7C%3FSoftware+Deployed%0D%0A+%7C%3FSoftware+Deployed.Developped+by+%28institutions%29%3DSoftware+Developer%0D%0A+%7C%3FDatasets+Used%0D%0A+%7C%3FDatasets+Used.Developed+by+Institution%3DDataset+Developer&eq=yes&p%5Bformat%5D=json&p%5Blimit%5D=500&p%5Boffset%5D=0&p%5Blink%5D=all&p%5Bheaders%5D=show&p%5Bmainlabel%5D=&p%5Bintro%5D=&p%5Boutro%5D=&p%5Bsearchlabel%5D=JSON&p%5Bdefault%5D=&p%5Btype%5D=full&p%5Bfilename%5D=result2.json&sort_num%5B%5D=&order_num%5B%5D=asc&sort_num%5B%5D=&order_num%5B%5D=asc&eq=yes'
 ```