roka https://robert.katzki.de Kirby Mon, 06 Apr 2020 00:00:00 +0200 Randomly ordered chaos. Personal website of Robert Katzki. Retro Radio https://robert.katzki.de/projects/retro-radio projects/retro-radio Mon, 06 Apr 2020 00:00:00 +0200

I cannot remember where I got this old radio from. Found it in a box and wanted to use it. As the selection of old school radio channels isn’t that great, I’ve build an internet radio into it.

It is based on a Raspberry Pi, using a Speaker Bonnet, an EInk display and the OnOff-Shim. I reused the old analogue volume control and on-off switch potentiometer. The sound comes from two different speakers to get the left and right channel and a wider sound frequence.

Setting the EInk display takes some time. Especially the shutdown isn’t the fastest as I used a display with color and that takes some extra time, but it doesn’t hurt while turning it off and walking away. It enabled me to have a nice off-message.

Currently it’s placed in our kitchen and the kids love to tune it to a kids radio station. I’m very happy with the result!

Check out the code over at Github.

Illustration while turned off
]]>
Thinking in React Hooks https://robert.katzki.de/posts/thinking-in-react-hooks posts/thinking-in-react-hooks Fri, 20 Mar 2020 00:00:00 +0100 ➭ Visit link

I just started work on a project with Hooks for the first time, so I needed to get into them now. Amelia Wattenberger created this beautiful intro. It finally made click for me on why to use them!

]]>
Hafencity & Speicherstadt https://robert.katzki.de/photos/2020/hafencity-speicherstadt photos/2020/hafencity-speicherstadt Mon, 09 Mar 2020 00:00:00 +0100

Spend some days of my parental leave strolling through the Hafencity and Speicherstadt in Hamburg. It’s always fascinating to find new perspectives.

]]>
Alster in the winter https://robert.katzki.de/photos/2020/alster-in-the-winter photos/2020/alster-in-the-winter Thu, 13 Feb 2020 00:00:00 +0100
]]>
Fusion Tables Archive https://robert.katzki.de/projects/fusiontables-archive projects/fusiontables-archive Wed, 30 Oct 2019 00:00:00 +0100

The Fusion Tables Archive is an easy way to preserve existing maps generated with Google Fusion Tables. It is a project by Google realized at Ubilabs.

An Node.js Express server running on AppEngine provides the export functionality. This server will export each selected Fusion Table to its own spreadsheet or CSV file “archive” in your Google Drive.

Exported datasets can be visualized on a Google Map. Its goal is to show that the exported data is save and looks similar to the original Fusion Tables dataset. Handling large datasets with thousands of entries and complex shapes was critical. This was achieved using deck.gl.

All the code of the exporter and the visualizer parts are open sourced.

My Role

I planned the architecture and built the server with Express, a Node.js framework. The export works completely without client side JavaScript to ensure that every user can export their data, no matter how old the browser is.

The visualizer part was implemented by me based on the previous work of Google and Uber to integrate deck.gl with Google Maps. It works as a static page, passing the data as URL parameters around. The goal was to provide a way to reuse the code for custom implementations and to prevent further lock-ins.

Impressions

Search for Fusion Tables.
Select any Fusion Tables to export.
Check the export progess.
An index sheet is created containing all the finished exports.
Large datasets with complex shapes work fine.
Hundreds of thousands data points work flawless.
Create an embed version to substitude old Fusion Tables embeds.
]]>
Denial https://robert.katzki.de/posts/denial posts/denial Thu, 19 Sep 2019 00:00:00 +0200 ➭ Visit link

I see this in myself. While being deeply worried, I'm also petrified. I don't know what to do. My commitment to change hasn't gone far enough yet. I'm part of the problem. I see my kids and want a healthy world for them. I want them to grow old without worrying. But I'm also stuck in my privileged way of living. Just one more steak, one more trip with the car, one more cheap purchase online. How bad can my impact be? After all, the system around me has to change. But that's just another excuse. I'm aware of that and I'm willing to do more than I'm currently doing. I often just feel lost.

The thoughts of Bastian Allgeier resonate with me. Recognizing the climate crisis is one thing. Changing oneself is hard. All things come with a cost. What is the right thing to do?

]]>
Visualize data on a Google Map with deck.gl https://robert.katzki.de/posts/visualize-data-on-a-google-map-with-deckgl posts/visualize-data-on-a-google-map-with-deckgl Thu, 05 Sep 2019 00:00:00 +0200 Visualizing large datasets on a map is not easy, you say? In this article I’ll show how to do just that in 160 lines of JavaScript using deck.gl on a Google Map, loading data from a Google Sheet.

We are going to use a large open dataset with information on power plants from all over the world, which should be interesting to visualize.

When we’re finished, the visualization will look like this:

Read on to discover how we build this map.

Get the data

First, grab the CSV data from the World Resources Institute and upload it to a Google Sheet.

As we won’t need all the data in the document, it can be slimmed down by removing unneeded columns. This reduces the amount of data the visualization has to load. We’ll focus on the data in the six columns you see below. You can find the example Sheet here and reuse it.

To show how power is produced, we will use the type of power plant to color a dot on the map. The size of the dot will be defined by the capacity of the power plant. Latitude and longitude will, of course, be used to place the dot on the map.

No native rendering of large datasets on Google Maps

There are almost 30.000 power plants in this dataset. Visualizing that much data on a Google Map is not without its problems. Using the Data Overlay provided by the Google Maps API is not very performant with that many items to render. Other methods such as using SVG as an overlay show similar performance problems with just a few hundreds of items already. So let’s take a look on deck.gl.

What is deck.gl?

deck.gl was published in 2016 and brought WebGL based rendering to maps, using the graphics card in our device. This is great news for us as it promises fast render performance! At first it didn’t work with Google Maps though. With the release of version 7 in April 2019, support for Google Maps was added and we’ll explore how easy it is to implement!

Of course deck.gl needs to be added to the visualization:

<script src="https://unpkg.com/deck.gl@7.0.9/dist.min.js"></script>

Create a Google Map

As a basis to render the data on, we create a Google Map. The Google Maps API is needed to create the map. Don’t forget to get an API key. It can be included like this:

<script src="https://maps.googleapis.com/maps/api/js?key=###YOUR_KEY###&callback=initMap"></script>

In the script tag, a callback is defined which will create the map when the API is loaded:

let map;

function initMap() {
  map = new google.maps.Map(document.getElementById('map'), {
    center: {lat: 17, lng: 0},
    minZoom: 3,
    zoom: 3,
    clickableIcons: false,
    disableDefaultUI: true,
    zoomControl: true
  });
}

Load data from the Google Sheet

As we have the data of the power plants in our Google Sheet, we need to load that data first. To be able to pull data from a Sheet, it needs to be published to the web. In the spreadsheet, go to “File” -> “Publish to the web” and click the publish button. The Sheet can now be consumed by our app!

To load it, a script tag is added to the page. Be sure to insert the ID of your sheet in the URL of the script tag. The ID can be found in the URL of your Sheet following the /d/ part. For our example the ID is 1MsFYOQlys_jyTACIZRbk3VWX9qaUdfrsr_r2Y-oxuZo.

At the end of the script tag, we define a callback that get’s called when the data is loaded:

<script
src="https://spreadsheets.google.com/feeds/list/###SHEET_ID###/1/public/values?alt=json-in-script&callback=createOverlay">
</script>

In the callback, we can inspect the data loaded:

function createOverlay(spreadsheetData) {
  console.log(spreadsheetData);
}

Create a deck.gl GeoJSON overlay

Google Sheets gives us a JSON in a strange nested structure. To render that data as GeoJSON on the map, we need to create a GeoJSON first in the createOverlay callback:

const data = {
  type: 'FeatureCollection',
  features: spreadsheetData.feed.entry.map(item => {
    return {
      type: 'Feature',
      geometry: {
        type: 'Point',
        coordinates: [
          Number(item.gsx$longitude.$t),
          Number(item.gsx$latitude.$t)
        ]
      },
      properties: {
        name: item.gsx$name.$t,
        country: item.gsx$countrylong.$t,
        capacity: Number(item.gsx$capacitymw.$t) || 0,
        primaryFuel: item.gsx$primaryfuel.$t
      }
    }
  })
};

Note that we add information on capacity and primary fuel to the properties so we can use it for the styling.

To add that GeoJSON to the map, we create a regular GeoJsonLayer from deck.gl:

const geojsonLayer = new GeoJsonLayer({
  id: 'geojsonLayer',
  data: data,
  pickable: true,
  pointRadiusMinPixels: 2,
  pointRadiusMaxPixels: 140,
  wrapLongitude: true,
  getRadius: d => d.properties.capacity * 40,
  getFillColor: d => fuelColorMapping[d.properties.primaryFuel] || [100, 100, 100, 194]
});

The GeoJSON we just created get’s passed in. To calculate the radius, we use the capacity from the properties. The color of the dot is defined by the primaryFuel. We are using a mapping object with the fuel type as the key and the color array as the value.

This is a layer now, but it’s still not on the map.

Add a deck.gl layer to a Google Map

Both the map and the layer need to be connected to render the data on the base map. deck.gl provides a GoogleMapsOverlay which does exactly that. Create one and assign it the map:

const overlay = new GoogleMapsOverlay({
  layers: [geojsonLayer]
});
overlay.setMap(map);

Yay! The data can now be seen on the map!

It’s really interesting to see the distribution of hydro power plants across the globe. The amount of coal power plants in China and India looks alarming with regard of the current climate crisis.

Show an infowindow on click

Seeing the data on the map is great, but getting more information on the capacity or the name of the power plant would be a nice addition. An infowindow helps with that:

const infowindow = new google.maps.InfoWindow({
  content: ''
});

map.addListener('click', event => {
  const picked = overlay._deck.pickObject({
    x: event.pixel.x,
    y: event.pixel.y,
    radius: 4,
    layerIds: ['geojsonLayer']
  });

  if (!picked) {
    infowindow.close();
    return;
  }

  infowindow.setContent(
    `<div>
      <div><b>${picked.object.properties.name}</b></div>
      <div>${picked.object.properties.country}</div>
      <div><b>capacity:</b> ${picked.object.properties.capacity}</div>
      <div><b>type:</b> ${picked.object.properties.primaryFuel}</div>
    </div>`
  );
  infowindow.setPosition({
    lng: picked.lngLat[0],
    lat: picked.lngLat[1]
  });
  infowindow.open(map);
});

When the map is clicked the overlay is checked for elements that can be picked at those coordinates. When none is found, close any open infowindow. Else, set the content to the data from the clicked power plants properties and open it at its location.

Looks good!

Conclusion

Loading data from a Google Sheet and creating a Google Map with a deck.gl GeoJsonLayer on top is possible with just a few lines of code. Creating visualizations from large datasets just became a lot easier and hopefully, this article helped you get started!

What are you going to visualize?

Be sure to check out the complete source code of the example. You can find the standalone example right here: deck-gl-on-google-maps.glitch.me.

This article was published first on the Ubilabs blog.

]]>
We Need Chrome No More https://robert.katzki.de/posts/we-need-chrome-no-more posts/we-need-chrome-no-more Wed, 21 Aug 2019 00:00:00 +0200 ➭ Visit link

Ten years ago, we needed Google Chrome to break the Web free from corporate hegemony, and we managed to do so for a brief period. Today, its dominance is stifling the very platform it once saved from the clutches of Microsoft. And no one, beside Google, needs that.

I'm really sad to see Microsoft switch to Chromium as the basis for their new Edge browser. The web is getting very homogen and that didn’t turn out to be a good idea in the past. Try Firefox for a free web. I use it since a few years and it got great over time.

Update 23.08.2019: Another great writeup by Blair Reeves touching this topic.

]]>
Browsers are pretty good at loading pages, it turns out https://robert.katzki.de/posts/browsers-are-pretty-good-at-loading-pages-it-turns-out posts/browsers-are-pretty-good-at-loading-pages-it-turns-out Thu, 25 Jul 2019 00:00:00 +0200 ➭ Visit link

Carter Sande writes about the current trend to use client-side navigation via JavaScript instead of relying on good old server rendered HTML pages linked via a classic <a> hyperlink.

It’s actually really hard to get it right—loading the new page is simple enough, but you also have to write code to display a loading bar, make the Back and Forward buttons work, show an error page if the connection drops, and so on.

The argument to do all this often is that it‘s faster to load just the content part than loading the complete page. Testing the old and new MDN site, he finds out:

Hang on a second! The JavaScript version wasn’t faster, it was way slower! What gives?

Browsers actually are pretty fast doing this. That‘s the reason I use server rendered HTML pages on my websites. Why complicate life for me and everyone else?

]]>
“Ethics” and Ethics https://robert.katzki.de/posts/ethics-and-ethics posts/ethics-and-ethics Fri, 14 Jun 2019 00:00:00 +0200 ➭ Visit link

Facebook fishing for our email passwords, Roomba is hovering up all the data on our homes, Amazon is listening to our conversations for laughs, Tik-Tok spying on our kids. And that we see so much dirt on the surface makes it likely that under the surface it’s even worse. The solution for all of this: “Ethics”.

I really enjoy the articles by iA. This one is about the “ethics” the tech industry applies and the difference to ethics.

]]>