security_vision/semantic_graph
1
0
Fork 1
Code Issues Pull requests Projects Releases 1 Wiki Activity

Compare commits

base: security_vision:main
Branches Tags
security_vision:main
security_vision:filtering
security_vision:v1
..
compare: security_vision:filtering
Branches Tags
security_vision:main
security_vision:filtering
security_vision:v1

No commits in common. "main" and "filtering" have entirely different histories.
main ... filtering

6 changed files with 183 additions and 595 deletions
Show stats Expand all files Collapse all files

33
README.md
View file

@ -22,6 +22,31 @@ python wiki_relations.py
_Ask_ SMW with the following query:
```
[[Category:Deployments||Institution]] OR [[Category:Technologies]] [[Developed by (institutions)::+]] OR [[Category:Technologies]] [[-Software Deployed::+]] OR [[Category:City]]
```
```
?Category
?Geolocation
?City
?City.Has Coordinates=City Coordinates
?City.Is in Country=City Country
?City.Is in Country.Has Coordinates=Country Coordinates
?Clients
?Managed by
?Used by
?Funded by
?Provided by
?Software Deployed
?Software Deployed.Developped by (institutions)=Software Developer
?Datasets Used
?Datasets Used.Developed by Institution=Dataset Developer
```
----
```
({{#ask: [[Category:Deployments||Institution]]
OR [[Category:Technologies]] [[Developed by (institutions)::+]] // TODO: + should give a subquery <q></q> with all institutions in EU
@ -50,16 +75,16 @@ OR [[Category:Technologies]] [[-Software Deployed::+]] // TODO: + should give a
|class=sortable wikitable smwtable
}}
TODO: see: https://www.semantic-mediawiki.org/wiki/Help:Querying_for_the_absence_of_a_property
{{#ask: [[Category:Deployments||Institution]] [[Is in Greens Report 2021::True]]
{{#ask: [[Category:Deployments||Institution]] [[Not in graph::!Greens Report 2021]]
OR [[Category:Technologies]] [[Developed by (institutions)::+]]
OR [[Category:Technologies]] [[-Software Deployed::+]]
|?Category
|?Geolocation
|?City
|?City.Has Coordinates=City Coordinates
|?City.Is in Country=City Country
|?City Country.Has Coordinates=Country Coordinates
|?Institution Type
|?Clients
|?Managed by
|?Used by
@ -69,8 +94,6 @@ TODO: see: https://www.semantic-mediawiki.org/wiki/Help:Querying_for_the_absence
|?Software Deployed.Developped by (institutions)=Software Developer
|?Datasets Used
|?Datasets Used.Developed by Institution=Dataset Developer
|?Related Institutions
|?Involved Entities
|format=broadtable
|limit=50
|offset=0

22
wiki_relations.py
View file

@ -4,7 +4,6 @@ import requests
import argparse
import datetime
import tqdm
import csv
@ -13,12 +12,11 @@ logger = logging.getLogger('wiki')
default_categories = [
# 'Person',
'Institution',
'Products',
'Technology',
'Deployments',
'Dataset',
'City',
#'Country',# for deployments without city we should configure Geolocation
'Technology Type',
# 'Country',# for deployments without city we should configure Geolocation
]
parser = argparse.ArgumentParser(description='Turn wiki into nodes & links, usable by d3-force.')
@ -30,8 +28,6 @@ parser.add_argument('--output', default="semantic_data.json",
help='Output JSON file')
parser.add_argument('--credentials', default="no_credentials.json",
help="JSON file containing the Bot's credentials")
parser.add_argument('--generate-csv', action='store_true',
help="generate edge.csv & nodes.csv")
args = parser.parse_args()
@ -261,17 +257,3 @@ if __name__ == "__main__":
with open(args.output, 'w') as fp:
json.dump(collection, fp)
if args.generate_csv:
with open('nodes.csv', 'w') as csvfile:
all_keys = set().union(*(d.keys() for d in collection['nodes']))
# all_keys = ['@id']
dict_writer = csv.DictWriter(csvfile, fieldnames=all_keys, extrasaction='ignore', restval='')
dict_writer.writeheader()
dict_writer.writerows(collection['nodes'])
with open('edges.csv', 'w') as csvfile:
all_keys = set().union(*(d.keys() for d in collection['links']))
dict_writer = csv.DictWriter(csvfile, fieldnames=all_keys, extrasaction='ignore', restval='')
dict_writer.writeheader()
dict_writer.writerows(collection['links'])

247
www/graph.css
View file

@ -7,13 +7,12 @@
:root {
--color1: #9741f9;
--color2: #f04a2d;
--color3: #0083ff;
--color2: #f3722c;
--color3: #f8961e;
/* --color4: #f9844a; */
--color5: #24C3B2;
--color6: #7bc748;
--color7: #e4a02b;/*#fff239;*/
--color7-2: #e4a02b;
--color5: #f9c74f;
--color6: #90be6d;
--color7: #43aa8b;
--color8: #4d908e;
--color9: #577590;
--color10: #277da1;
@ -23,13 +22,11 @@
--selected-color: var(--color1);
--link-color: rgba(255,255,255,0.5);
--link-hover-color: var(--hover-color);
--link-hover-related-color: var(--hover-related-color);
--link-hover-related-color: var(--hover-related-color);
--link-focus-color: var(--hover-color);
--body-back: #96a7b7; /*#9cb3c9; /*#8195a7; /*#9cb3c9; #b9cada*/
--title-color: #1c1c1c;
--zoom: 1; /* to be overriden by js */
}
body {
@ -66,41 +63,33 @@ svg .links line, svg .links path {
/* stroke: #f3722c; */
/* stroke: #9df32c; */
stroke: var(--link-color);
stroke-width: calc(12px / var(--zoom));
stroke-width: 6;
fill: none;
/* transition: stroke-width 1s; */
transition: stroke-width 1s;
cursor: pointer;
stroke-linecap: round;
}
svg .selectedNode .links line, svg .selectedNode .links path {
opacity: .3;
}
svg .links line.hover, svg .links path.hover {
stroke: var(--link-hover-color) !important;
opacity: 1;
stroke: var(--link-hover-color);
/* stroke-width: 12; */
/* marker-end: url(#arrowHeadSelected) !important; */
marker-end: url(#arrowHeadSelected);
}
svg .links .linkedHover path{
stroke: var(--link-hover-related-color);
opacity: 1;
stroke-width: calc(12px / var(--zoom));
/* marker-end: url(#arrowHeadSelectedRelated); */
stroke-width: 12;
marker-end: url(#arrowHeadSelectedRelated);
}
svg .links .linkedSelected path{
stroke: var(--link-hover-related-color);
opacity: 1;
stroke-width: calc(12px / var(--zoom));
/* marker-end: url(#arrowHeadSelectedRelated); */
stroke: var(--link-focus-color);
stroke-width: 12;
marker-end: url(#arrowHeadSelected);
}
/* svg.zoomed .links line, svg.zoomed .links path {
svg.zoomed .links line, svg.zoomed .links path {
stroke-width: 2;
} */
}
/* svg.zoomed .links line, svg.zoomed .links path.hover {
stroke-width: 4;
@ -165,18 +154,16 @@ svg #header text#subtitle {
cursor: pointer;
}
.node text.nodeTitle {
text-anchor: start;
dominant-baseline: hanging;
font-size: calc(20pt / var(--zoom)) !important;
/*achieves a 'text-anchor: top'*/
/* font-size: 16pt; */
/*Set this in JS*/
transition: opacity 1s;
fill: black; /*also when hovering node*/
transition: font-size .4s, opacity 1s;
fill: #5d5d5f; /*also when hovering node*/
opacity: 1;
/* pointer-events: none; */
pointer-events: none;
/*prevent mouse glitches*/
}
@ -184,13 +171,6 @@ svg #header text#subtitle {
.node text.nodeTitle.overlapping {
/* used to be shown on hover, but disabled now that we have a tooltip */
opacity: 0;
pointer-events: none;
}
.selectedNode .node:not(.linkedSelected):not(.selected) text.nodeTitle{
/* used to be shown on hover, but disabled now that we have a tooltip */
opacity: 0;
pointer-events: none;
}
svg.zoomed .node text.nodeTitle {
@ -203,12 +183,6 @@ svg.zoomed.zoomed2 .node text.nodeTitle {
.node circle, .node path {
fill: lightgray;
transform: scale(calc(1.5 / var(--zoom-sqrt)));
}
#filters .node circle, #filters .node path {
transform: none;
}
/* Whenever a connected link is hovered */
@ -216,14 +190,10 @@ svg.zoomed.zoomed2 .node text.nodeTitle {
.node.linkHover circle, .node.linkHover path, .node.linkedHover path, label:hover .node path {
fill: var(--hover-related-color) !important;
stroke: var(--hover-related-color);
stroke-width: 2px;
stroke-width: 5px;
}
.node.linkHover text, .node.linkedHover text{
.node.linkedSelected path {
fill: var(--hover-related-color) !important;
}
.selectedNode .node:not(.linkedSelected) path {
fill: lightgray !important;
opacity: .5;
/* same as linkHover/linkedHover but without border */
}
@ -234,12 +204,7 @@ svg.zoomed.zoomed2 .node text.nodeTitle {
.node:hover circle, .node:hover path, .node.selected path {
fill: var(--hover-color) !important;
stroke: var(--hover-color);
stroke-width: 2px;
}
.node:hover text, .node.selected text {
fill: var(--hover-color) !important;
stroke-width: 5px;
}
/*
.node:hover text {
@ -247,11 +212,7 @@ svg.zoomed.zoomed2 .node text.nodeTitle {
fill: var(--hover-color);
} */
/* .selectedNode .node circle, .selectedNode .node path {
fill: lightgray !important;
} */
.selectedNode .node.selected circle, .selectedNode .node.selected path {
.node.selected circle, .node.selected path {
fill: var(--selected-color) !important;
stroke: var(--selected-color);
stroke-width: 5px;
@ -284,8 +245,7 @@ svg.zoomed.zoomed2 .node text.nodeTitle {
.node.Institution circle, .node.Institution path {
/* fill: lightcoral; */
/* fill: #11F999; */
fill: darkgray;
fill: red;
}
.node.Dataset circle, .node.Dataset path {
@ -297,49 +257,6 @@ svg.zoomed.zoomed2 .node text.nodeTitle {
opacity: 1 !important;
}
.node.Institution.Institution-company path, .node.Institution.Institution-company text{
fill: var(--color2);
}
.node.Institution.Institution-government path, .node.Institution.Institution-government text{
fill: var(--color3);
}
.node.Institution.Institution-local-government path{
fill: #75F06D;
}
.node.Institution.Institution-law-enforcement path{
fill: var(--color7);
}
.node.Institution.Institution-law-enforcement text{
fill: var(--color7-2);
}
.node.Institution.Institution-ngo path, .node.Institution.Institution-ngo text{
fill: var(--color5);
}
.node.Institution.Institution-university path{
fill: #A6E447;
}
.node.Institution.Institution-research path, .node.Institution.Institution-research text{
fill: var(--color6);
}
.node.Institution.Institution-project path{
fill: #C1DB31;
}
.node.Institution.Institution-watchdog path{
fill: #CED628;
}
.node.Institution.Institution-expert-group path{
fill: #DAD121;
}
.node.Institution.Institution-foundation path{
fill: #E5CB1E;
}
.node.Institution.Institution-international-organization path{
fill: gray;
}
.node.Institution.Institution-art-project path{
/* fill: blue; */
}
/* .node.Person circle {
fill: var(--color2)
}
@ -391,17 +308,6 @@ svg.zoomed.zoomed2 .node text.nodeTitle {
border-radius: 5px;
box-shadow: 2px 2px 5px rgba(0, 0, 0, .5);
}
#tooltip.deploymentTooltip{
background-color: black;
color: white;
}
#tooltip.link{
}
#tooltip .entity{
font-weight: bold;
}
#tooltip:not(.visible){
position:absolute;
@ -419,9 +325,6 @@ svg.zoomed.zoomed2 .node text.nodeTitle {
color: black;
text-align: center;;
}
#tooltip.deploymentTooltip .category{
color: white;
}
#tooltip .category::before{
content:'· '
}
@ -433,12 +336,6 @@ svg.zoomed.zoomed2 .node text.nodeTitle {
color: gray;
text-align: center;;
}
#tooltip .node_sources::before{
content: 'source ';
}
#tooltip .node_sources{
text-align: center;
}
#closeInfo {
cursor: pointer;
@ -466,7 +363,6 @@ header {
background: white;
padding: 10px;
border-top-left-radius: 5px;
min-width: 300px;
}
h1 {
@ -491,7 +387,7 @@ p.subtitle {
#filters label {
cursor: pointer;
display: block;
padding: 5px 10px;
padding: 10px;
}
#filters label svg {
@ -522,43 +418,6 @@ p.subtitle {
/* box-shadow: inset 0 0 5px black; */
}
#filters .institution_types{
margin-left: 30px;
}
#filter-items.filter-institution .institution_types{
/* display:none */
color: darkgray;
pointer-events: none;
}
#filter-items.filter-institution .institution_types span{
text-decoration: line-through;
}
#filters h3{
cursor: pointer;
}
#filters h3::before{
display: inline-block;
float: left;
content: '\fe3f'; /*⬆*/
transform: rotate(180deg);
transition: transform .8s;
}
#filters.hide h3::before{
transform: rotate(0deg);
}
#filter-items{
max-height: 1000px;
transition: max-height .8s;
}
#filters.hide #filter-items{
max-height: 0px;
}
#map .borders {
stroke-width: 6px;
stroke: rgb(221, 210, 210);
@ -575,57 +434,3 @@ p.subtitle {
font-size: 30;
}
#closeSelection{
pointer-events: none;
position: fixed;
bottom: -100px;
left: 0;
background-color: white;
font-size: 50px;
width: 80px;
height: 80px;
text-align:center;
z-index: 10;
line-height: 80px;
transition: bottom .2s;
}
.selectedNode #closeSelection{
pointer-events: all;
cursor: pointer;
bottom: 0;
}
#sources{
position: fixed;
bottom: 0;
left: 80px;
height: 47px;
padding: 20px 20px 13px 20px;
background: black;
color: white;
transition: bottom .2s;
line-height: 40px;
}
#sources:not(.visible){
bottom:-100px;
}
#sources h3{
margin: 5px 0;
text-align: left;
display: inline-block;
font-size: 100%;
}
#sources h3::after{
content:':';
}
#sources .node_sources{
display: inline;
}
#sources a{
color: lightblue;
}

457
www/graph.js
View file

@ -4,9 +4,9 @@ const CONFIG = {
'subtitle': "A survey of the European Union",
// 'nodeSize': 8,
'nodeRadius': 5,
'nodeRepositionPadding': 12,
'nodeRepositionPadding': 10,
'baseUrl': 'https://www.securityvision.io/wiki/index.php/',
'dataUrl': 'remote_biometric_identification.json',
'dataUrl': 'result2.json',
'preSimulate': false, // run simulation before starting, so we don't start with lines jumping around
'labels': {
'rotate': true,
@ -17,92 +17,7 @@ const CONFIG = {
'lonMax': 35,
'center': [11, 47],
},
"institution_map": {
"Local Government": "Government",
"Regional Government": "Government",
"International Organization": "Government",
"Watchdog": "Government",
// "NGO": "Organisation",
"Foundation": "NGO",
"University": "Research",
"Expert Group": "Research",
},
"filters": {
"Institution": {
"label": "Institution",
"type": "categories",
},
// TODO: nested filters
// TODO:restructure, allow for groups of types:
// [12:08, 07-05-2021] Francesco Ragazzi: Government / Regional / local can be the same
// [12:08, 07-05-2021] Francesco Ragazzi: Also IO
// [12:08, 07-05-2021] Francesco Ragazzi: NGO, Foundation can be grouped
// [12:08, 07-05-2021] Francesco Ragazzi: University, Research, Expert group can be grouped
// [12:08, 07-05-2021] Francesco Ragazzi: Watchdog and can also be grouped with government
"Government": {
// "label": "Institution",
"type": "institution_types",
},
"Law Enforcement": {
// "label": "Institution",
"type": "institution_types",
},
"Company": {
// "label": "Institution",
"type": "institution_types",
},
"NGO": {
// "label": "Institution",
"type": "institution_types",
},
// "Regional Government": {
// // "label": "Institution",
// "type": "institution_types",
// },
// "Local Government": {
// // "label": "Institution",
// "type": "institution_types",
// },
// "Foundation": {
// // "label": "Institution",
// "type": "institution_types",
// },
// "University": {
// // "label": "Institution",
// "type": "institution_types",
// },
"Research": {
// "label": "Institution",
"type": "institution_types",
},
// "Labour Union": {
// // "label": "Institution",
// "type": "institution_types",
// },
// "Watchdog": {
// // "label": "Institution",
// "type": "institution_types",
// },
// "Expert Group": {
// // "label": "Institution",
// "type": "institution_types",
// },
// "International Organization": {
// // "label": "Institution",
// "type": "institution_types",
// },
// "Art Project": {
// // "label": "Institution",
// "type": "institution_types",
// },
"Deployments": {
"label": "Deployment",
"type": "categories",
},
/* "Technology", "Dataset"*/
},
"filters": ["Institution", "Deployments",/* "Technology", "Dataset"*/],
"link_properties": [
"Clients",
@ -113,53 +28,8 @@ const CONFIG = {
"Software Deployed",
"Software Developer",
"Dataset Developer",
"Related Institutions",
"Involved Entities",
],
"link_labels": {
"Clients": {
"label": "is client of",
"swap": true,
},
"Managed by": {
"label": "manages",
"swap": true,
},
"Used by": {
"label": "uses",
"swap": true,
},
"Funded by": {
"label": "is funded by",
"swap": false,
},
"Provided by": {
"label": "provides",
"swap": true,
},
"Software Deployed": {
"label": "is used in",
"swap": true,
},
"Software Developer": {
"label": "is developed by",
"swap": false,
},
"Dataset Developer": {
"label": "develops dataset for",
"swap": true,
},
"Related Institutions": {
"label": "is related to",
"swap": true,
},
"Involved Entities": {
"label": "is involved in",
"swap": true,
},
},
"geo_properties": [
"Geolocation",
"City Coordinates",
@ -176,7 +46,7 @@ const CONFIG = {
"Country": ["Country"],
"City": ["City"],
// ["Deployment type"], // TODO: select this
"Institution": ["Institution Type"], // TODO: select this (local gov, etc.)
// ["Institution type"], // TODO: select this (local gov, etc.)
"Dataset": ["Datasets used"],
"Company": ["Managed by", "Provided by", "Developped by (institutions)"],
"Tech": ["Technologies Used", "Software Deployed"],
@ -184,7 +54,7 @@ const CONFIG = {
},
"zoom": {
"scale_min": .2,
"scale_max": 10,
"scale_max": 20,
},
"cases": [
@ -193,17 +63,6 @@ const CONFIG = {
]
};
function getLinkLabelConfig(linkName){
if(CONFIG.link_labels.hasOwnProperty(linkName)){
return CONFIG.link_labels[linkName];
}
return {
'label': linkName,
'swap': false,
}
}
// let width = window.innerWidth;
// let height = window.innerHeight;
@ -337,17 +196,7 @@ function getTitle(obj) {
}
function getCategories(obj) {
// console.log(obj);
let cats = obj.printouts['Category'].map(n => n.fulltext.split(':')[1]);
if (obj.printouts.hasOwnProperty("Institution Type") && obj.printouts['Institution Type'].length) {
obj.printouts['Institution Type'].forEach(type => {
cats.push(getInstitutionClass(type.fulltext));
});
}
// return
return cats;
}
function getInstitutionClass(name) {
return "Institution-" + slugify(name);
return obj.printouts['Category'].map(n => n.fulltext.split(':')[1]);
}
function getClasses(obj) {
const classes = getCategories(obj);
@ -366,10 +215,7 @@ class NodeMap {
this.root = d3.select(parent);
this.resizeEvent = window.addEventListener('resize', this.resize.bind(this));
this.tooltipEl = document.getElementById('tooltip');
this.sourcesEl = document.getElementById('sources');
this.selectedNode = null;
document.getElementById('closeSelection').addEventListener('click', (ev) => this.deselectNode());
}
resize() {
@ -389,16 +235,8 @@ class NodeMap {
this.render();
}
// calculate which text labels overlap.
calculateLabels() {
let els;
if(this.selectedNode === null) {
els = document.querySelectorAll('.node text')
} else {
// only related texts are visible
// so only these need to be considered
els = document.querySelectorAll('.node.linkedSelected text, .node.selected text')
}
const els = document.querySelectorAll('.node text')
for (let i = 0; i < els.length; i++) {
const el = els[i];
let overlapping = false;
@ -425,7 +263,7 @@ class NodeMap {
render() {
this.svg = this.root.append('svg')
this.svg.append('defs').html(`<marker markerHeight="3" markerWidth="3" refY="0" refX="6" viewBox="0 -3 8 6" preserveAspectRatio="none" orient="auto" id="arrowHead" ><path d="M0,-3L8,0L0,3"></path></marker><marker markerHeight="3" markerWidth="3" refY="0" refX="6" viewBox="0 -3 8 6" preserveAspectRatio="none" orient="auto" id="arrowHeadSelected"><path d="M0,-3L8,0L0,3"></path></marker><marker markerHeight="3" markerWidth="3" refY="0" refX="6" viewBox="0 -3 8 6" preserveAspectRatio="none" orient="auto" id="arrowHeadSelectedRelated"><path d="M0,-3L8,0L0,3"></path></marker>
this.svg.append('defs').html(`<marker markerHeight="4" markerWidth="4" refY="0" refX="6" viewBox="0 -3 8 6" preserveAspectRatio="none" orient="auto" id="arrowHead" ><path d="M0,-3L8,0L0,3"></path></marker><marker markerHeight="4" markerWidth="4" refY="0" refX="6" viewBox="0 -3 8 6" preserveAspectRatio="none" orient="auto" id="arrowHeadSelected"><path d="M0,-3L8,0L0,3"></path></marker><marker markerHeight="4" markerWidth="4" refY="0" refX="6" viewBox="0 -3 8 6" preserveAspectRatio="none" orient="auto" id="arrowHeadSelectedRelated"><path d="M0,-3L8,0L0,3"></path></marker>
<!--Sketching:-->
<defs>
<filter id="tint">
@ -444,7 +282,7 @@ class NodeMap {
</filter>
</defs>`);
this.svg.on('click', (e) => { console.log(e); this.deselectNode() })
this.svg.on('click', (e) => { console.log(e); this.deselectNode()})
// const noise = 0.001;
// this.svg.append('defs').append('filter').attr('id', 'splotch').html( `${
@ -507,7 +345,7 @@ class NodeMap {
return "country";
})
.attr("d", this.proj)
// .attr("filter", 'url(#splotch)')
// .attr("filter", 'url(#splotch)')
// .attr("fill", );
this.g_borders
@ -529,8 +367,6 @@ class NodeMap {
if (zoomTimeout) {
clearTimeout(zoomTimeout)
}
document.querySelector(':root').style.setProperty('--zoom', evt.transform.k);
document.querySelector(':root').style.setProperty('--zoom-sqrt', Math.pow(evt.transform.k, 1/3));
zoomTimeout = setTimeout(() => {
this.g_nodes.attr('style', `font-size:${22000 / this.height / evt.transform.k}pt`)
setTimeout(() => {
@ -605,6 +441,7 @@ class NodeMap {
this.graph.nodes.forEach((d) => {
for (const prop of CONFIG.geo_properties) {
// console.log(this,d.printouts, prop)
if (d.printouts[prop].length) {
// console.log("fix node", d);
@ -754,24 +591,10 @@ class NodeMap {
}
showTooltip(el, node, links) {
if(el.tagName != 'path'){
let parentEl = el.parentNode;
if(parentEl.tagName != 'g'){
parentEl = parentEl.parentNode;
}
el = parentEl.querySelector('path');
}
const categories = getCategories(node);
if (categories.includes('Deployments')){
this.tooltipEl.classList.add('deploymentTooltip');
} else {
this.tooltipEl.classList.remove('deploymentTooltip');
}
// TODO: make links optional (otherwise collect links here)
this.tooltipEl.innerHTML = `
<span class='category'>${categories[0]}</span>
<span class='category'>${getCategories(node)[0]}</span>
<h3>${node.fulltext}</h3>
`;
if (links.length) {
@ -780,54 +603,17 @@ class NodeMap {
<span class='clickForMore'>Click to examine ${links.length} ${rels}</span>
`;
}
const rect = el.getBoundingClientRect()
const rectTT = this.tooltipEl.getBoundingClientRect();
this.tooltipEl.style.top = (rect.top - rectTT.height) + 'px';
this.tooltipEl.style.left = (rect.left + rect.width / 2 - rectTT.width / 2) + 'px';
// console.log(el, node, rect.top);
this.tooltipEl.classList.add('visible', 'node');
}
showRelationTooltip(link, evt) {
const {label, swap} = getLinkLabelConfig(link.name);
this.tooltipEl.classList.remove('deploymentTooltip');
if(swap){
this.tooltipEl.innerHTML = `
<span class='relation'>
<span class='entity'>${link.target.fulltext}</span>
${label}
<span class='entity'>${link.source.fulltext}</span>
</span>
`;
} else {
this.tooltipEl.innerHTML = `
<span class='relation'>
<span class='entity'>${link.source.fulltext}</span>
${label}
<span class='entity'>${link.target.fulltext}</span>
</span>
`;
}
const rectTT = this.tooltipEl.getBoundingClientRect();
this.trackerEv = (evt) => {
this.tooltipEl.style.top = (evt.clientY - rectTT.height - 10) + 'px';
this.tooltipEl.style.left = (evt.clientX - rectTT.width / 2) + 'px';
};
window.addEventListener('mousemove', this.trackerEv);
this.tooltipEl.classList.add('visible', 'link');
this.tooltipEl.classList.add('visible');
}
hideTooltip() {
this.tooltipEl.classList.remove('visible', 'node', 'link');
if(this.trackerEv){
window.removeEventListener('mousemove', this.trackerEv);
this.trackerEv = null;
}
this.tooltipEl.classList.remove('visible');
}
selectNode(node) {
@ -851,19 +637,14 @@ class NodeMap {
document.getElementById(node.id).classList.add('selected');
connectedNodes.forEach(n => document.getElementById(n.id).classList.add('linkedSelected'));
links.forEach(l => document.getElementById(getLinkId(l)).classList.add('linkedSelected'));
this.container.classed('selectedNode', true);
document.body.classList.add('selectedNode');
this.showSources(node);
// TODO: show details;
// alert('not yet implemented');
}
deselectNode() {
this.hideSources();
this.selectedNode = null;
let nodeEls = document.getElementsByClassName('selected');
while (nodeEls.length) {
@ -873,39 +654,6 @@ class NodeMap {
while (els.length) {
els[0].classList.remove('linkedSelected');
}
this.container.classed('selectedNode', false);
document.body.classList.remove('selectedNode');
}
showSources(node){
const categories = getCategories(node);
if (!categories.includes('Deployments')){
return;
}
if(!node.printouts['Source'].length) {
return;
}
setTimeout(() => { // give potential visible sources time to hide
let sources = [];
for(let source of node.printouts['Source']){
const url = document.createElement('a');
url.href = source;
const hostname = url.hostname.startsWith('www.') ? url.hostname.substring(4) : url.hostname;
sources .push(`<a href="${source}" target="_blank">${hostname}</a>`);
}
const title = node.printouts['Source'].length > 1 ? "Sources" : "Source"
this.sourcesEl.innerHTML = `<h3>${title}</h3> ` + sources.join(', ');
this.sourcesEl.classList.add('visible');
}, 500);
}
hideSources(){
this.sourcesEl.classList.remove('visible');
}
update() {
@ -916,7 +664,6 @@ class NodeMap {
// see also: https://www.createwithdata.com/enter-exit-with-d3-join/
this.node = this.node.data(this.graph.nodes, d => d.id)
.join((enter) => {
// let group = enter.insert("g", ":first-child")
let group = enter.append("g")
.attr("class", getClasses)
.attr("id", (n) => getIdForTitle(n.fulltext));
@ -956,10 +703,7 @@ class NodeMap {
.attr('d', (n) => {
return getSymbolForNode(n)(n);
})
var nodeTitle = group.append('text').attr("class", "nodeTitle").attr("y", "4").attr('x', 5);
// nodeTitle.on('mouseover', (evt, n) =>{
// console.log(evt,n)
// });
var nodeTitle = group.append('text').attr("class", "nodeTitle").attr("y", "3").attr('x', 5);
nodeTitle
.each(function (node, i, nodes) {
var textLength = void 0;
@ -970,7 +714,7 @@ class NodeMap {
titleTexts = splitText(titleText);
}
if (titleTexts !== false) {
const tspan1 = self.append("tspan").text(titleTexts[0]).attr("y", "4").attr("x", "5");
const tspan1 = self.append("tspan").text(titleTexts[0]).attr("y", "3").attr("x", "5");
const tspan = self.append("tspan").text(titleTexts[1]).attr("dy", "1em").attr("x", "5");
// const textLength1 = tspan.node().getComputedTextLength();
// const textLength2 = tspan.node().getComputedTextLength();
@ -985,6 +729,64 @@ class NodeMap {
// const labelPadding = 1;
// // // the component used to render each label
// var textLabel = fc.layoutTextLabel()
// .padding(labelPadding)
// //.value(function(d) { return map_data.properties.iso; });
// //.value(function(d) { return d.properties.iso; });
// .value( (d) => getTitle(d));
// // a strategy that combines simulated annealing with removal
// // of overlapping labels
// // */fc.layoutGreedy
// const strategy = fc.layoutRemoveOverlaps(fc.layoutGreedy());
// // create the layout that positions the labels
// this.layoutLabels = fc.layoutLabel(strategy)
// .size((d, i, g) => {
// // measure the label and add the required padding
// const textSize = g[i].getElementsByTagName('text')[0].getBBox();
// console.log(textSize);
// // return [30, 20];
// return [textSize.width + labelPadding * 2, textSize.height + labelPadding * 2];
// })
// .position(d => [d.x, d.y])
// .component(textLabel);
// // render!
// // this.node.datum(this.graph.nodes,).call(labels)
// this.labels = this.container.append('g').attr('class','labels');
// this.labels.datum(this.graph.nodes)
// // // this.node
// .call(this.layoutLabels);
// // use simulate annealing to find minimum overlapping text label positions
// //https://github.com/d3fc/d3fc-label-layout/blob/master/README.md
// var strategy = fc.layoutGreedy();
// //var strategy = fc.layoutAnnealing();
// // create the layout that positions the labels
// var labels = fc.layoutLabel(strategy)
// .size(function (_, i, g) {
// // measure the label and add the required padding
// var textSize = d3.select(g[i])
// .select('text')
// .node()
// .getBBox();
// return [textSize.width + labelPadding * 2, textSize.height + labelPadding * 2];
// })
// .position((d) => this.projection([d.lon, d.lat]); })
// .component(textLabel);
// // render!
// this.container.datum(countries)
// .call(labels);
this.link = this.link
.data(this.graph.links)
.join(
@ -993,10 +795,10 @@ class NodeMap {
.attr("class", (l) => "link " + slugify(l.name))
.attr("id", getLinkId);
group.append("path")
// .attr("marker-end", "url(#arrowHead)")
.attr("marker-end", "url(#arrowHead)")
.attr('id', (d, i) => 'linkpath_' + i)
.on("mouseover", (ev, link) => {
d3.select(ev.target).classed('hover', true);
.on("mouseover", function (ev, link) {
d3.select(this).classed('hover', true);
const nodes = document.getElementsByClassName('node');
for (let n of nodes) {
const d = d3.select(n).datum();
@ -1004,15 +806,17 @@ class NodeMap {
n.classList.add('linkHover');
}
}
this.showRelationTooltip(link, ev);
}).on("mouseout", (ev, link) => {
this.hideTooltip();
d3.select(ev.target).classed('hover', false);
// console.log(l);
}).on("mouseout", function (ev, link) {
d3.select(this).classed('hover', false);
const nodes = document.getElementsByClassName('linkHover');
while (nodes.length) {
nodes[0].classList.remove('linkHover');
}
// l.classed('hover',false);
// l.target.classed('hover',false);
// l.source.classed('hover',false);
// console.log(l,'l');
}).on("click", (ev, link) => {
ev.stopPropagation();
this.selectNode(link.source);
@ -1085,25 +889,28 @@ class NodeMap {
var tgtSize = _mapGraph.getSizeForNode(l.target);
// Compute the line endpoint such that the arrow
// it not in the center, but rather slightly out of it
// use a small ofset for the angle to compensate roughly for the curve
l.sourceX = sourceX + Math.sin(angle+.5) * srcSize;
l.targetX = targetX - Math.sin(angle-.5) * tgtSize;
l.sourceY = l.source.y + Math.cos(angle+.5) * srcSize;
l.targetY = l.target.y - Math.cos(angle-.5) * tgtSize;
// is touching the edge of the node rectangle perfectly.
l.sourceX = sourceX + Math.sin(angle) * srcSize;
l.targetX = targetX - Math.sin(angle) * tgtSize;
l.sourceY = l.source.y + Math.cos(angle) * srcSize;
l.targetY = l.target.y - Math.cos(angle) * tgtSize;
// const coor_source = _mapGraph.projection.invert([l.source.x, l.source.y]);
// const coor_target = _mapGraph.projection.invert([l.target.x, l.target.y]);
// const middleCoor = [coor_source[0] * .5 + coor_target[0] * .5, coor_source[1] * .5 + coor_target[1] * .5];
// const middlePoint = _mapGraph.projection(middleCoor);
// find radius of arc based on distance between points
const dr = Math.sqrt(dx * dx + dy * dy);
// "M" + d.source.x + "," + d.source.y + "A" + dr + "," + dr + " 0 0,1 " + d.target.x + "," + d.target.y
let rel = d3.select(this);
rel.select("path")
rel.select("path") //${middlePoint[0]},${middlePoint[1]}
// .attr('d', `M ${l.sourceX},${l.sourceY} L ${l.targetX},${l.targetY}`)
.attr('d', `M ${l.sourceX},${l.sourceY} A ${dr},${dr} 0 0,1 ${l.targetX},${l.targetY}`)
// .attr("x1", l.sourceX)
// .attr("y1", l.sourceY)
// .attr("x2", l.targetX)
// .attr("y2", l.targetY)
rel.select('text')
.attr("transform", function (d) {
@ -1170,7 +977,6 @@ class NodeMap {
}
}
// zoom & translate the graph to fit the provided nodes
zoomFit(nodes, paddingPercent = 0.8, transitionDuration = 2000) {
// var bounds = root.node().getBBox();
const x0 = Math.min(...nodes.map(n => n.x - CONFIG.nodeRadius));
@ -1238,7 +1044,7 @@ class AlluvialMap {
render() {
this.svg = this.root.append('svg')
this.resize();
this.sankey = d3.sankey()
@ -1368,17 +1174,6 @@ JsonToGraph = function (data) {
i++;
let node = data.results[node_id];
node.id = getIdForTitle(node.fulltext); //node_id;
// group institution types
if(node.printouts['Institution Type'].length ){
for (let idx = 0; idx < node.printouts['Institution Type'].length; idx++) {
const type = node.printouts['Institution Type'][idx];
if(CONFIG.institution_map.hasOwnProperty(type.fulltext)){
node.printouts['Institution Type'][idx].fulltext = CONFIG.institution_map[type.fulltext];
}
}
// node.printouts['Institution Type'] = CONFIG.institution_map[node.printouts['Institution Type']];
}
nodes.push(node);
// console.log(node_id, node);
@ -1649,9 +1444,10 @@ class Store {
this.filters = {
'categories': [],
'institution_types': [],
}
this.filter();
}
@ -1707,18 +1503,10 @@ class Store {
});
}
isFiltered(node) {
if (this.filters.categories.includes(node.printouts['Category'][0].fulltext.split(':')[1]))
return true;
if (node.printouts['Institution Type'].length && this.filters.institution_types.includes(node.printouts['Institution Type'][0].fulltext))
return true;
return false;
}
filter() {
// add and remove nodes from data based on type filters
this.nodes.forEach((n) => {
if (!this.isFiltered(n)) {
if (!this.filters.categories.includes(n.printouts['Category'][0].fulltext.split(':')[1])) {
if (n.filtered || typeof n.filtered === 'undefined') {
n.filtered = false;
this.graph.nodes.push(n);
@ -1736,16 +1524,14 @@ class Store {
// add and remove links from data based on availability of nodes
this.links.forEach((l) => {
// if (this.graph.nodes.includes(l.source) && this.graph.nodes.includes(l.target)) {
if (!l.source.filtered && !l.target.filtered) {
if (this.graph.nodes.includes(l.source) && this.graph.nodes.includes(l.target)) {
if (l.filtered || typeof l.filtered === 'undefined')
this.graph.links.push(l);
l.filtered = false;
} else {
if (l.filtered === false) {
console.log('filter', l.id);
this.graph.links.forEach((d, i) => {
if (l.nr === d.nr) {
if (l.id === d.id) {
this.graph.links.splice(i, 1);
}
});
@ -1753,55 +1539,40 @@ class Store {
l.filtered = true;
}
});
console.log(this.graph.nodes.length, this.graph.links.length)
}
render() {
Object.keys(CONFIG.filters).forEach(f => {
const settings = CONFIG.filters[f];
CONFIG.filters.forEach(f => {
// if (settings.type == 'institution_types')
// // TODO; For now, skip
// return;
let categories = [f];
if (settings.type == 'institution_types')
categories = ['Institution', getInstitutionClass(f)];
let labelEl = document.createElement('label');
labelEl.setAttribute('id', 'filter-'+slugify(f));
labelEl.classList.add(settings.type)
let labelEl = document.createElement('label')
let inputEl = document.createElement('input')
let textEl = document.createElement('span');
let svg = d3.select(labelEl).append('svg')
.attr("viewBox", [-12, -12, 24, 24]);
svg.append('g')
.attr("class", "node " + categories.join(' '))
.attr("class", "node " + f)
.append('path')
.attr('d', getSymbolForCategories(categories)());
.attr('d', getSymbolForCategories(f)());
inputEl.type = "checkbox";
textEl.innerText = settings.hasOwnProperty('label') ? settings.label : f;
textEl.innerText = f;
labelEl.appendChild(inputEl);
labelEl.appendChild(textEl);
if (!this.filters[settings.type].includes(f)) {
if (!this.filters.categories.includes(f)) {
inputEl.checked = true;
}
inputEl.addEventListener('change', (e) => {
if (e.target.checked) {
this.filters[settings.type].forEach((d, i) => {
this.filters.categories.forEach((d, i) => {
if (d == f) {
this.filters[settings.type].splice(i, 1);
this.filters.categories.splice(i, 1);
}
});
this.root.classList.remove('filter-'+slugify(f))
} else {
if (!this.filters[settings.type].includes(f)) {
this.filters[settings.type].push(f);
if (!this.filters.categories.includes(f)) {
this.filters.categories.push(f);
}
this.root.classList.add('filter-'+slugify(f))
}
this.filter();
this.update();
@ -1826,7 +1597,7 @@ Promise.all([fetch(req_data), fetch(req_world)])
})
.then(([data, world]) => {
var graph = JsonToGraph(data);
var store = new Store(graph, '#filter-items');
var store = new Store(graph, '#filters');
mapGraph.setWorld(world);
mapGraph.setStore(store);

17
www/index.html
View file

@ -3,23 +3,22 @@
<head>
<meta charset="utf-8">
<link rel="stylesheet" href="graph.css">
<title>Remote Biometric Identification | A survey of the European Union</title>
</head>
<body>
<div id='tooltip'></div>
<div id='sources'></div>
<div id='closeSelection'>&times;</div>
<div id='map'></div>
<!-- <div id='alluvial'></div> -->
<header>
<h1>Remote Biometric Identification</h1>
<p class='subtitle'>A survey of the European Union</p>
<aside id="filters">
<h3 onclick="this.parentNode.classList.toggle('hide');">Filter</h3>
<div id="filter-items"></div>
<h3>Legend</h3>
</aside>
</header>
@ -29,8 +28,16 @@
<script src="https://d3js.org/d3-geo-projection.v3.min.js"></script>
<script src="https://d3js.org/topojson.v3.min.js"></script>
<script src="//unpkg.com/d3-geo-zoom"></script>
<script src="https://unpkg.com/d3-sankey@0"></script>
<!-- <script src="//unpkg.com/d3fc@14.0.1"></script> -->
<script src="graph.js"></script>
<script>
if(window.location.hash == '#light') {
document.body.classList.add('light');
}
</script>
</body>
</html>

2
www/story.html
View file

@ -150,7 +150,7 @@
Quisque vulputate odio neque, eget laoreet felis molestie sit amet.
</p>
</div>
<div class='section' data-title="Burglary-Free Neighbourhood">
<div class='section' data-title="Data-lab Burglary-free Neighbourhood">
<h2>Burglary-free Neighbourhood</h2>
<p>
Aenean fringilla nisl sed ante congue, in commodo tellus ullamcorper. Nullam sollicitudin, lacus id