Feed aggregator

KnackForge: How to update Drupal 8 core?

Planet Drupal -

How to update Drupal 8 core?

Let's see how to update your Drupal site between 8.x.x minor and patch versions. For example, from 8.1.2 to 8.1.3, or from 8.3.5 to 8.4.0. I hope this will help you.

  • If you are upgrading to Drupal version x.y.z

           x -> is known as the major version number

           y -> is known as the minor version number

           z -> is known as the patch version number.

Sat, 03/24/2018 - 10:31

You Can Finally Get More Page Rules For 5 For $5

Cloudflare Blog -

Since CloudFlare launched Page Rules in 2012, our Free, Pro and Business users have been asking for a way to get more Page Rules without committing to the next plan up. Starting today, anyone on CloudFlare can add 5 additional Page Rules for just $5/month.

Page Rules allows you to fine tune your site speed and to apply CloudFlare’s wide range of features to specific parts of your site. Page Rules are also accessible over our API, so you can integrate them into your build process or sync them across your domains.

To help you get the most out of Page Rules, we’re also launching a tutorial site that features videos to help you setup Page Rules for specific content management systems like WordPress, Magento and Drupal, and for specific goals like optimizing your website's speed, increasing security, and saving on your bandwidth costs.

ActiveLAMP: Incrementally Upgrading to Drupal 8

Planet Drupal -

In our last video we answered the question should I build on Drupal 7 or Drupal 8? We mentioned that we have a site currently in production on both Drupal 7 and Drupal 8. To the end-user browsing the site, it is completely transparent, they have no idea they are hitting two different Drupal instances. Today, we are going to show you how to pull this off.

Read more...

How the Consumer Product Safety Commission is (Inadvertently) Behind the Internet’s Largest DDoS Attacks

Cloudflare Blog -

The mission of the United State's Government's Consumer Product Safety Commission (CPSC) is to protect consumers from injury by products. It's ironic then that the CPSC is playing an unwitting role in most of the largest DDoS attacks seen on the Internet. To understand how, you need to understand a bit about how you launch a high volume DDoS.

Logo of the Consumer Product Safety Commission

Amplification

DDoS attacks are inherently about an attacker sending more traffic to a victim than the victim can handle. The challenge for an attacker is to find a way to generate a large amount of traffic. Launching a DDoS attack is a criminal act, so an attacker can't simply go sign up for large transit contracts. Instead, attackers find ways to leverage other people's resources.

One of the most effective strategies is known as an amplification attack. In these attacks, an attacker can amplify their resources by reflecting them off other resources online that magnify the level of traffic. The most popular amplification vector is known as DNS reflection.

DNS Reflection

We've written about DNS reflection attacks in detail before. The basics are that an attacker generates DNS requests from a network that allows source IP address spoofing. The attacker forges the victim's IP as the source of the DNS requests. The attacker sends these requests to DNS resolvers that aren't locked down and respond to requests from any network. The DNS resolvers receive the requests and then send the responses back to the spoofed IP of the victim.

This technique "amplifies" an attack because a small DNS request will generate a larger response. How much larger depends on the size of the DNS record that's being queried. And here's where the Consumer Product Safety Commission comes in.

#1 Among DDoS Attackers

You see, the CPSC’s DNS server will respond with a very large DNS record when sent a very small DNS query. This DNS query is only 65 bytes:

dig ANY cpsc.gov +notcp +bufsize=65535 @X.X.X.X

Substitute X.X.X.X for one of the 13 million currently open DNS resolvers and you'll get back a response that is 4,426 bytes. To generate 68Gbps of traffic to a victim requires only 1Gbps of requests with a spoofed source IP address to open resolvers for the CPSC.gov DNS record. That's an amplification factor of 68x.

Now, lots of people have large DNS records, but the CPSC.gov record is very large and particularly popular among attackers. Based on both data about the large volumetric attacks we see hitting CloudFlare, as well as data from various resolver honey pots we run, approximately 94% of the DNS reflection attack requests we see are for CPSC.gov. (The next most popular are httrack.com which is used by 2.9% of attacker requests and isc.org which is used by 0.5%.)

At one level there's nothing the CPSC can do to prevent attackers from using their DNS record to launch attacks. CPSC's resources aren't being queried directly by the attackers so they can't block the requests themselves. However, they could construct their DNS record more carefully to make it smaller and therefore less attractive to attackers.

To see how, we can walk through the CPSC DNS record entry by entry and make recommendations on how to reduce its size.

The CPSC's Ginormous Zone

Here’s the CPSC.gov's entire 4,426-byte DNS response which you get from running the ANY DNS query above against an open resolver (scroll to the left in the grey boxes below to see the full DNS record):

cpsc.gov. 18894 IN SOA auth00.ns.uu.net. hostmaster.uu.net. 994622 1800 600 1728000 21600 cpsc.gov. 18894 IN A 63.74.109.2 cpsc.gov. 18894 IN AAAA 2600:803:240::2 cpsc.gov. 18894 IN NS auth61.ns.uu.net. cpsc.gov. 18894 IN NS auth00.ns.uu.net. cpsc.gov. 18894 IN MX 5 stagg.cpsc.gov. cpsc.gov. 18894 IN MX 5 hormel.cpsc.gov. cpsc.gov. 18894 IN TXT "v=spf1 ip4:63.74.109.6 ip4:63.74.109.10 ip4:63.74.109.20 mx a:list.cpsc.gov -all" cpsc.gov. 18894 IN DNSKEY 256 3 7 AwEAAbpeSszphwwkOIJn1ha6DE/W3YRXFR2vsMi0RKhq5x9t487UJc0c eamz5TZj6KV5/tzL8/Qr2jnTaQmpWtJHbnF0kqpxeZIR+wzaNbMtEH30 UF5BDv9Bya0W9I+40dS48996kedhEvL6KwmMelB7FH6QPd0ixyhp0+ci 5vew9lzTESEsJ2X2uJrCqo3UacsHyYIzaTSXPpfwizQCql4VySq6+im1 74QaYw/FU4aADAv3R2KQvsR/uI0a7oOihxDDAvtYG7SZvotW3ASZFscd 4B6Yd84RMZC3yGdGtyrSD6tZsiJZoXhLQkkf0kOTWCjPvD8oPm+yYiGI Cm64eM3kflU= cpsc.gov. 18894 IN DNSKEY 256 3 7 AwEAAXQUfP1/CdN5/YYXxWdePx3dWhhY7RmzxEfGXSZ0ea5BZoOXTLHd giWlm9ORnZ5hC+kaDRoYjgZXnkZOkhQhCDBwm2O2IOGBjBLMMtbm9hNK b2WGE8WC/E3j56YfepaMSzhxICu1xgY8JeYhmfpc3C5Z9Mm2oPm91cwU WZZY8i5f2F04tNBBymXTfu0mytCvp/dNxUjM45svY+SNRJltgcy07qBj T/GHDglEc6iJdBtvik3Nd4RfFfI+ftG8xSxfna3Nv4BVdYxPkE4us3ti 0dv/Ejwl9kuoXhT7/Ydpdze/boWmIuwjn3a66Afg7CHtmYyW6InLz57r tzUTGUdStgc= cpsc.gov. 18894 IN DNSKEY 257 3 7 AwEAAZztz17cVspxUk8egfYEFLuyPXVETlPdT2PAuy+cZTk3afTS7cda Tnsk43AIqgnCkTvHE9m4gVuOhNmFjPIABPkfmaCtOzyqVmLjxb36JMxJ TnhBPBYjWY0HrBdEGCGG7eZzY4ll9kAMPIxE1OmMl9iM0dQSzamITEWN 89oPptHnlbjz8k7nQO3xyzXreamjhIW/2iIJhM+CdHe2CgMhPtf8b4QR 8CuIBMH07gvsTKljuvQLiS1ThQYYpmLgriiWjnQFum2FJe6J7x8joDAq YCzbQUdGSyJPp6FYibaG70Y62fIf9DNgHRMH/3c79DW9RmwzFggjfKLf y4hOgRbsVFc= cpsc.gov. 18894 IN DNSKEY 257 3 7 AwEAAX5Tor9V7TnhfUMAL67reT+IFYd+4ciQv/UnvZbNgj7DgDuJPpcl Owh6ypAldCYgTXkF2Qt+an9WVp+Khsp2wRCCOhvGIUR9sOGdzxumDUCT Uru2dxHAqIn1QYSjuT8huMDDyBJmnoA4AY1Te86mcE1Jwpo+S9KoB23Z JgnMedU+6i8Qm9cdGLNM7nqEXhgKgmKc/387UFdh25jltsg0d2gOK//q k2HfLdDqv8XlrlacfMsSXniVwK7E6mtqcfbF518M2bl6UFJWXuxp+cU8 0WdmGiQfxmLvm62a2aVs9IzR6qGg0Ce5bxbx68v6gYTgIOUBm8ERytZ3 T2jzc0QOKQc= cpsc.gov. 18894 IN NSEC3PARAM 1 0 12 AABBCCDD cpsc.gov. 18894 IN RRSIG NS 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. e4NrJJq0j7ZcHb/UAm0E34nuPfnbgCW6FARJM/5QKwgYDqZpCixaVQgL s9KGLR/eYQsp+BdPQibdLD3Ef2ICqQmBl0qRIyxh5Sbg7XiXnObJXgtR +vJuhVjzLtYfAGGicAm3MfYzcxk2/Usr8rwr/EakzhMLEr03Tshj4uwm WaRQ97M3R6dyTDvJ35X0m76KeWAyIS/Z2WmbGzNWUn8qpgerD8d1CIrP R7S5psgTMFIApD5I6mfJjuvZ5B+RRs7VhjY9tn6IKKoBLMqHxZAtwf1n td9Ho8anT3LJrrB1b/NZsRWhVdGLDlohDKzM9qqa6wQkoHg+/zrRrapZ 5NiUlQ== cpsc.gov. 18894 IN RRSIG MX 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. jAYmQYaSMOkU7VfwTqIb9CHO3nBBjkiYQ4zOoRIaD8u0ASMaweXhy/H2 75kxrhvwYs9VR0RXOlTal69uV9vN74cffu4lIfi9mIfSSVvo3JZz6dDM bpgmpdXRd6DEFVpmrHfTUK1sbKGB5qo3Yxiz38VzcZr7hclApmC/Bf0/ /luymk7BeCvtGjLVamRaaHKD76f/FA+0dkLvNOZITqrj36wZwOb1JTv8 XAl61dVxJl3R3z8TCtWm8+Za5nTzt3oNjad3Hf4g9r+88ugCvCcOibSU ujUH1sdc3/Uvr7ZEGT+51a6H6BkoctWwAimkN6CwbM0M8GhqwNiISXov 5QKO+Q== cpsc.gov. 18894 IN RRSIG TXT 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. CYoU8McQOKcy82SKfTXPfT8AnheHQmtdf4m0se/Ht7q64ll6iflSkh3O VGRmrQlgiMdAKaPIRMU7liGftMUntYKDu7RivVG18JokePDYHH9vYjss nQf92Reu08dOwm/Ica9cM1naGL1RDzvBbDkQkucyjJe2fvd00Npj+vBG H5kyLULmbkfD2PVL6c5F+eEhyGCF/IKPveag1ymkwBMGRM+Za+LYOeLY SkB1zwL5GodnHj/uaZfkGxT5ulNOnukMx536719pjf7ig/oMdaf8MySo hGQCjseS5ON6dRYuAzkOVDwdc5/Qw2jWqQlQUUH213AAYKz6eZXZtLAt +AflUQ== cpsc.gov. 18894 IN RRSIG A 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. DOLz8p1/pyr3LByKjHKZEcrgQwDqkB+Gg3nlcqSbFp8ChIenAjBGBvXy 24u7atI55ZxTqF6dtQEo3+D9a30piWeDm5U2tOg/VTHtwFFfi2h5DwQx WNxh6/71h7uzoidiBr4cKEkZSjB4F+bRviqJomxSdGPA9ZrQFZv4RcCb B8InJJZwyu9gfT767nt3yOq+Os/nlr19vyicmt5MiG9WQpxBn/ZhUG2u DYf6WkZVVo2i3nrD9PmpOjeDuj4g0DKAlhyPzPM5L8CKNDJSxGAdAHy+ auOJUFRPan3TGQqmgaqaEs/8HLiRr9QSgafuRJCO0HNW97p91zXX2krI AtDHgw== cpsc.gov. 18894 IN RRSIG AAAA 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. ANnB5Mw+iU6b2TD/3ULnt2LR138Vvd80BaYuOX765TNxpsoPosTxbls1 n6810qVvkHxR+uFafmrHA9PHOwJavEgmwSk4TyKbiTdu5fY0+wnR0tXr Fl8ARAALvmunyZ49aaEQUa0TUgKyNQIKoNktnfiLFhcFpsfQD678FGm3 IKtFnFpj3HvM3Y3vkfKORnqHEyT37O8EK5cIbK4YZ3UE/AJwqBEqUt7x xca51gRK3IW0/MBRoargRbg1zgVxhNCnnxWk59uMEER0wePsQK9XAMJ1 9QjpXlPFW+0426pFqbUTQyymwTjHFu2G8yzJas5g2r+oQShPTR80K6WG rmNbCA== cpsc.gov. 18894 IN RRSIG SOA 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. I7umZbpbUnh24bvphYP9K70imTyDFxc5QgOEDVVCSkmtVMhRaOU0eiSO 6b/O3ExcMhL0xn+D+/Xs1R7z9zPsKrzvGCfqotkxu8L1mRpAmjgr21WH mEzXftcx+IbSrOCwOa7YuPyBGqJW3f8kb/UHcI7ykqk08xvglD2I77PV TgI59oYsZyufRhfChcFxEyhBY2LZ4o/UO5Tw8Lg5HH/9OGtA0FHum1To v/CeLBOaATrZNygZINKmHlsP6Mm8j83jM4bRNxLGPs8VjpDbKVAHtROq aXR0Zy81zNs0eWfs4vThdOf56nCUnA/HiwlVNgmbO219CCMKQxoBBuY+ FYcWfA== cpsc.gov. 18894 IN RRSIG DNSKEY 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. HlTGZb9hSEwtxLayRdqxBkS+1jZIx/2FbsGaNJREfijFf7HTX/gpmXil g2DcVSvXwh6J473YKKpvdIe0LOfgIGA9NWaO5M3FVeKbwRk4OENdKTEG Vwm/uB8Av0aLsxD8YRvprPKxi7YxVDta7/cRMX1vP4ULQHrAbrJ5xlbu SzrlwK5b1BDoil3qfPNVbpHg2NMuqyp/hbFzUfWzvRNbLG3PJZfUEppl QTzInHnBONKmS0oRW4oUj8la2Zu6BGIOjB/0IktrLyxRKHJ2KNZDa1+o 0pKr68/gY2j7EfJwWrrohieG5LHfisCdTgaqs1zZbYgv+CAAl9hvq6vJ 5xrFxA== cpsc.gov. 18894 IN RRSIG DNSKEY 7 2 21600 20160824030507 20160817020507 58273 cpsc.gov. DlWHCC0re6XY3+MzRDYpJ2nIo2bWczGkJpxvA1yXhUivz8Qlv5mK1mtV 3YTislUzE9qWQTDOMoatcmQHVqZiHZG9d7w8/5XzBv3YuAqJNvUKs5QF 7YFBfF/acQCI40/pEyDhwNn5NbCJ3kpJBo7KSHBJpo9GNMnVQMmcK6bo UCeXEeI9Z1cnu/9XiD8jVSyVEMSs0KdjDmKEXNXgChVUWCr6+oWn8g/T 1SCUHF5mWn7r/ZgGPAyBwzoJMA/f9qdEBighiOz/DNe5dSKHAr/3Cf78 AJIFvEPe+Yn+KsRo14WLzzfuNT2NZp/nQ6TIHVLC04uDMXeimjQiHMJZ 8QDQzw== cpsc.gov. 18894 IN RRSIG NSEC3PARAM 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. ink6+sysDZdHK+1Ntfllouw2I7nAvqiV+9uPQI7O0MFdGMSIZJ6Dli7v z81S+yTp3diXZv+m4j0wTFSGtcN3Zte14jC+Kl+SJ2SRmfXedgcAXcZW QrPYgDps2C4jxxytxePKxHuDKLuSsxtUFmonOWKJuC2p7UqGDhsa5AHS XB8SoUX5ezgWjrSZgUmA7sWDE8szlvOtv1DtJKR62RIXQEOckd7z2HKh oQe5fZx1C1eoR+Ppgi1eNiC+Dqh3Q4FYcCzvSfY/rtZ88dyg8DKj0tV1 RMXYDR8+x18qSqAK9NlXLldumVTKBcC3rzTUKd0PLj5/sdbyaYo6X2mn RpHQXg== cpsc.gov. 893 IN RRSIG DS 8 2 3600 20160824041014 20160817041014 64415 gov. c78BIiQZ0L8C19OaNWlRtmr1/2cnL9yap0tcC4WPgKTtzluj4esJaN7u PZkKgaQME0BuImPs94s/IKy0JmWrobApCWPVjhtHq0E4m8UWsBbaORJX Ihg8JuS80rHFdPU6Y4u6Gu2ShTmE5eM+0wE1L5KKjt2co+Wa7AN4W2Kv Udo= cpsc.gov. 893 IN DS 58273 7 1 75F81A67F1D76F017B02689E9586FCF3A68F655F cpsc.gov. 893 IN DS 58273 7 2 A04919AA51B4395014E2753430A6DD6AF7585CE2C5C2E693E9ADD4DE C1365DC9 cpsc.gov. 893 IN DS 27793 7 2 C078BB7D7A46B0654A7D83755BA6FAB6AA56DAC4CB4D94B6A3562B67 BA7C285C cpsc.gov. 893 IN DS 27793 7 1 E148C892A62ABF2B9157293DA4EE270D9880C238

You may get the records back in a different order, but you should see something similar if you run the same ANY query.

Walking through the DNS zone file line-by-line you can see the mistakes that the CPSC has made that bloat its record and learn a bit about DNS and, in particular, DNSSEC.

The Good

The Good, The Bad, and The Ugly artwork by Billy Perkins

Let's start with the good. Here are the first eight records, which are pretty standard and can't be optimized much.

cpsc.gov. 18894 IN SOA auth00.ns.uu.net. hostmaster.uu.net. 994622 1800 600 1728000 21600 cpsc.gov. 18894 IN A 63.74.109.2 cpsc.gov. 18894 IN AAAA 2600:803:240::2 cpsc.gov. 18894 IN NS auth61.ns.uu.net. cpsc.gov. 18894 IN NS auth00.ns.uu.net. cpsc.gov. 18894 IN MX 5 stagg.cpsc.gov. cpsc.gov. 18894 IN MX 5 hormel.cpsc.gov. cpsc.gov. 18894 IN TXT "v=spf1 ip4:63.74.109.6 ip4:63.74.109.10 ip4:63.74.109.20 mx a:list.cpsc.gov -all"

These first eight records are bread and butter DNS: SOA, A, AAAA, NS, MX, and TXT. If you want a quick refresher on these basic DNS records, read on. Otherwise, you can skip to the next section ("The Bad") to see where the DNSSEC mess begins.

The first record is the SOA record. SOA stands for Start of Authority. Walking through the record left-to-right, the number 18894 — which you'll see appears next to all records — is the TTL (Time to Live) at the moment for the particular resolver I queried. That number represents the seconds until the resolver needs to fetch another result from authoritative DNS server. If I queried the same resolver 10 seconds later, the number would be 18884 — 10 seconds less than it had been before. If you query a different resolver you'll likely get another number that is somewhere between 21600 (the max TTL the CPSC.gov has specified) and 0. When the number reaches 0, the resolver will query the authoritative name server, update its cache, and reset the TTL to 21600 — the maximum TTL specified for the record.

cpsc.gov. 18894 IN SOA auth00.ns.uu.net. hostmaster.uu.net. 994622 1800 600 1728000 21600

Continuing on the SOA record, next up is "IN" — which also appears on every DNS record in the CPSC zone. IN stands for "Internet Class." DNS predated the Internet, so there are other classes that are possible, but IN is the only class you'll see in common use today. After "IN" comes "SOA" which designates this record type.

The next entries are the contents of the SOA record. The first entry specifies the primary authoritative name server for the domain ("auth00.ns.uu.net."). The second is the email address of the name server's administrator ("hostmaster.uu.net." means [email protected] is the name server's administrator's email). "994622" is the zone serial number, which other name servers use to make sure they're in sync with the primary authoritative name server. "1800" is the refresh interval, specifying the number of seconds before non-primary name servers should check to see if the zone has changed. "600" is the retry interval, specifying the number of seconds to wait if a refresh failed. "1728000" is the number of seconds that non-primary name servers can serve a zone if they've failed to reach the primary name server. In practice with modern DNS setups, none of these numbers end up having much of an impact.

The last number in the SOA record, however, is important. "21600" is the length of time in seconds a negative result should be cached by recursive resolvers. For example, if you query "12345.cpsc.gov" — a record that doesn't exist — then the resolver you query will store the fact that the record doesn't exist for 21600 seconds.

SOA records look complicated but, in practice, only the last number — specifying the number of seconds that a negative result should be cached — has an impact on modern DNS implementations.

The other records in this group of 8 are less cryptic than SOA so we can quickly work through how they work.

cpsc.gov. 18894 IN A 63.74.109.2 cpsc.gov. 18894 IN AAAA 2600:803:240::2 cpsc.gov. 18894 IN NS auth61.ns.uu.net. cpsc.gov. 18894 IN NS auth00.ns.uu.net.

The A and AAAA records are "anchor" records which specify the IP addresses responsible for the domain. A records are for IPv4 addresses — in this case 63.74.109.2 — and AAAA records are for IPv6 addresses — in this case 2600:803:240::2. The two NS entries define the name servers responsible for the domain ("auth00.ns.uu.net" which is primary as specified by the SOA record, and "auth61.ns.uu.net" which is secondary).

cpsc.gov. 18894 IN MX 5 stagg.cpsc.gov. cpsc.gov. 18894 IN MX 5 hormel.cpsc.gov.

The two MX records specify the mail servers responsible for email sent to the domain ("hormel.cpsc.gov" and "stagg.cpsc.gov" — someone at the CPSC seems to have a sense of humor as Hormel is the maker of the pork product spam and "Stagg" is a brand of chili also made by Hormel). The number "5" in both MX records is the weight of that mail server. The lower the number the more it will be preferred. Since these are equally weighted, email will be load balanced equally between the two.

cpsc.gov. 18894 IN TXT "v=spf1 ip4:63.74.109.6 ip4:63.74.109.10 ip4:63.74.109.20 mx a:list.cpsc.gov -all"

Not much from the above could be optimized. The next record is special kind of TXT record known as a SPF record. It specifies what domains or IPs are allowed to send email on behalf of CPSC.gov. In this case, three IP addresses (63.74.109.6, 63.74.109.10 & 63.74.109.20), the MX records (Hormel & Stagg), and the A record for the subdomain (lists.cpsc.gov). The "-all" at the end means mail sent from other domains or IPs not specified here should fail. Alternatives could be "~all" which would soft-fail mail sent from other domains/IPs, or "+all" which would allow mail to be sent from any domain/IP — which would effectively render the SPF record useless.

There's a tiny optimization that CPSC.gov could make by listing one CIDR (63.74.109.0/24) rather than each of the three IPs. By doing so they'd save 15 bytes:

CPSC.gov. 18894 IN TXT "v=spf1 ip4:63.74.109.0/24 mx a:list.cpsc.gov -all"

However, as you'll see, that's hardly worth the effort compared with other optimizations they could make.

The Bad

The Good, The Bad, and The Ugly artwork by Billy Perkins

The misconfiguration of the CPSC.gov DNS largely involves how they've setup DNSSEC. DNSSEC is the mechanism to sign DNS records and help prevent cache poisoning. Each record in the DNS zone needs to be signed. To accomplish this you need RRSIG records and a simple change in their configuration could cut the size of the DDoS attacks the CPSC is indirectly responsible for nearly in half.

The signatures for each record are included as RRSIG records. Here's an example of the RRSIG record signing CPSC.gov's SOA record (again, scroll to the left in the box to see the whole record):

cpsc.gov. 18894 IN RRSIG SOA 7 2 21600 20160824030507 20160817020507 53799 cpsc.gov. I7umZbpbUnh24bvphYP9K70imTyDFxc5QgOEDVVCSkmtVMhRaOU0eiSO 6b/O3ExcMhL0xn+D+/Xs1R7z9zPsKrzvGCfqotkxu8L1mRpAmjgr21WH mEzXftcx+IbSrOCwOa7YuPyBGqJW3f8kb/UHcI7ykqk08xvglD2I77PV TgI59oYsZyufRhfChcFxEyhBY2LZ4o/UO5Tw8Lg5HH/9OGtA0FHum1To v/CeLBOaATrZNygZINKmHlsP6Mm8j83jM4bRNxLGPs8VjpDbKVAHtROq aXR0Zy81zNs0eWfs4vThdOf56nCUnA/HiwlVNgmbO219CCMKQxoBBuY+ FYcWfA==

Let's walk through this RRSIG record. Just like with the previous records "18894" is the TTL in seconds, "IN" stands for "Internet Class", and RRSIG is the type of DNS record. "SOA" here specifies the type of record this signature applies to. That's easy. However, after that, things get cryptic (both literally and metaphorically).

The "7" here specifies the signature algorithm being used. By the DNSSEC specification, the 7th zone signing algorithm is: RSASHA1-NSEC3-SHA1.

Here's a list of the 10 current algorithms (represented by 14 numbers, just to be confusing) that can be used for signing public DNS records along with the number — or range of numbers if the signatures can be of variable lengths — of bytes each signature generates:

  1. RSA/MD5 / 128–512 bytes (deprecated)
  2. [Not used for RRSIG records]
  3. DSA/SHA1 / 41 bytes
  4. [Reserved]
  5. RSA/SHA-1 / 128–512 bytes
  6. DSA-NSEC3-SHA1 / 328 bits
  7. RSASHA1-NSEC3-SHA1 / 128–512 bytes
  8. RSA/SHA-256 / 128–512 bytes
  9. [Reserved]
  10. RSA/SHA-512 / 128–512 bytes
  11. [Reserved]
  12. GOST R 34.10-2001 / 64 bytes
  13. ECDSA Curve P-256 with SHA-256 / 64 bytes
  14. ECDSA Curve P-384 with SHA-384 / 96 bytes

Algorithm 7 allows the signer to specify a key length between 128 and 512 bytes. The CPSC specifies a key length of 256 bytes, which, today, for RSA is considered secure. Unfortunately, that means they have a 256-byte signature for every RRSIG record. Compare that with Algorithm 13 which, for a higher level of security, would produce only a 64-byte signature — one fourth the size per RRSIG record.

Returning to the RRSIG record, the "2" (after the "7") represents the number of "labels" in the domain being signed. In this case, the domain is cpsc.gov, which is 2 labels deep. If it were www.cpsc.gov then it would be 3 labels. a.b.cpsc.gov would be 4 labels.

"21600" represents the TTL that was used when the signature was originally calculated. Since the TTL could be different for each resolver's response, the TTL value needs to be set to a default to ensure that the signature is calculated consistently.

The next two numbers are timestamps. The first (20160824030507) represents the time this signature will expire (24 August 2016 at 03:05:07 UTC). The second (20160817020507) represents the time this signature became valid (17 August 2016 at 02:05:07 UTC).

"53799" is a checksum on the DNSKEY that was used to sign this record. This checksum is used to more quickly find the DNSKEY that was used to sign this particular record when more than one key is returned. (More on DNSKEYs in the next section.)

After the checksum is "cpsc.gov." which is the root domain. This root domain is included regardless of the number of subdomains (e.g., cpsc.gov, www.cpsc.gov, and a.b.cpsc.gov would all have "cpsc.gov." listed as the root domain). This specifies where to find the DNSKEY records that could have signed this record.

Next up is the signature itself. In this case, the signature is encoded in base64 and 256 bytes long. The base64 encoding means that the 256 bytes turns into 344 on the wire.

Stepping back, the easy optimization here for the RRSIG record would be to pick a better signature algorithm. By choosing Algorithm 13, rather than Algorithm 7, the CPSC could save 192 bytes per RRSIG record. We've written previously about why we use Algorithm 13 for CloudFlare's DNSSEC implementation.
Doing so here would add up because there are ten RRSIG records in the CPSC's zone in order to sign each of the other records (e.g., A, AAAA, NS, MX, etc...).

Just by switching to a better encryption algorithm the CPSC could increase the security of their DNS zone and save 1,920 bytes per DNS response. That small change alone would cut the maximum possible DNS reflection attack possible with the CPSC's zone by 44 percent.

While that's the biggest savings, choosing a less optimal outcome can't be considered an actual mistake. Unfortunately, the CPSC's zone has some ugly mistakes as well.

The Ugly

The Good, The Bad, and The Ugly artwork by Billy Perkins

The RRSIG records discussed in the previous section need to be verified with a public key. These public keys are stored as DNSKEY records. For the CPSC, there are four DNSKEYs that each look like this:

cpsc.gov. 18894 IN DNSKEY 256 3 7 AwEAAbpeSszphwwkOIJn1ha6DE/W3YRXFR2vsMi0RKhq5x9t487UJc0c eamz5TZj6KV5/tzL8/Qr2jnTaQmpWtJHbnF0kqpxeZIR+wzaNbMtEH30 UF5BDv9Bya0W9I+40dS48996kedhEvL6KwmMelB7FH6QPd0ixyhp0+ci 5vew9lzTESEsJ2X2uJrCqo3UacsHyYIzaTSXPpfwizQCql4VySq6+im1 74QaYw/FU4aADAv3R2KQvsR/uI0a7oOihxDDAvtYG7SZvotW3ASZFscd 4B6Yd84RMZC3yGdGtyrSD6tZsiJZoXhLQkkf0kOTWCjPvD8oPm+yYiGI Cm64eM3kflU=

Walking through the record they're similarly formatted to RRSIG records. Once again, 18894 is the TTL on the record, the record type is DNSKEY and that's followed by three numbers: 256, 3 and 7. These are the flags field, protocol, and algorithm.

The flags field is 16-bits wide representing a number of potential boolean flags. Only two bits of this 16-bit field are currently assigned for use with the rest reserved for the future. A value of 256 (or 0000000010000000 or only the 9th flag set to true) indicates that this DNSKEY holds a key that can be used to verify RRSIGs. The protocol requires a 16-bit number so there's no optimization possible here.

The protocol field is 3. That is the only valid value for this field in a DNSKEY record. As the RFC says:

The Protocol Field MUST have value 3, and the DNSKEY RR MUST be treated as invalid during signature verification if it is found to be some value other than 3.

Again, that seems a bit arbitrary but it's what the protocol requires so there's no optimization.

Finally, the algorithm field is 7 which indicates RSASHA1-NSEC3-SHA1. So, it should be an RSA public key formatted according to RFC3110.

Passing it through a base64 decoder and hexdump we can take a look inside:

00000000 03 01 00 01 ba 5e 4a cc e9 87 0c 24 38 82 67 d6 |.....^J....$8.g.| 00000010 16 ba 0c 4f d6 dd 84 57 15 1d af b0 c8 b4 44 a8 |...O...W......D.| 00000020 6a e7 1f 6d e3 ce d4 25 cd 1c 79 a9 b3 e5 36 63 |j..m...%..y...6c| 00000030 e8 a5 79 fe dc cb f3 f4 2b da 39 d3 69 09 a9 5a |..y.....+.9.i..Z| 00000040 d2 47 6e 71 74 92 aa 71 79 92 11 fb 0c da 35 b3 |.Gnqt..qy.....5.| 00000050 2d 10 7d f4 50 5e 41 0e ff 41 c9 ad 16 f4 8f b8 |-.}.P^A..A......| 00000060 d1 d4 b8 f3 df 7a 91 e7 61 12 f2 fa 2b 09 8c 7a |.....z..a...+..z| 00000070 50 7b 14 7e 90 3d dd 22 c7 28 69 d3 e7 22 e6 f7 |P{.~.=.".(i.."..| 00000080 b0 f6 5c d3 11 21 2c 27 65 f6 b8 9a c2 aa 8d d4 |..\..!,'e.......| 00000090 69 cb 07 c9 82 33 69 34 97 3e 97 f0 8b 34 02 aa |i....3i4.>...4..| 000000a0 5e 15 c9 2a ba fa 29 b5 ef 84 1a 63 0f c5 53 86 |^..*..)....c..S.| 000000b0 80 0c 0b f7 47 62 90 be c4 7f b8 8d 1a ee 83 a2 |....Gb..........| 000000c0 87 10 c3 02 fb 58 1b b4 99 be 8b 56 dc 04 99 16 |.....X.....V....| 000000d0 c7 1d e0 1e 98 77 ce 11 31 90 b7 c8 67 46 b7 2a |.....w..1...gF.*| 000000e0 d2 0f ab 59 b2 22 59 a1 78 4b 42 49 1f d2 43 93 |...Y."Y.xKBI..C.| 000000f0 58 28 cf bc 3f 28 3e 6f b2 62 21 88 0a 6e b8 78 |X(..?(>o.b!..n.x| 00000100 cd e4 7e 55 |..~U|

Notice that the first four bytes are 03 01 00 01. That indicates an exponent of 65537 (03 means three bytes of exponent, 010001 in hex is 65537) which is very common for the RSA algorithm. The rest of the record consists of 256 bytes (1,024 bits). These 1,024 bits are the modulus part of the RSA key. Thus this is a 1,024 bit key.

Like with RRSIG, the CPSC could have better security and a shorter record if they specified a more efficient algorithm for their DNSKEY records. Using Algorithm 13 would save 192 bytes per DNSKEY, or 768 bytes across the 4 records.

But there's the rub: the CPSC doesn't need to have 4 DNSKEY records at all. Except during a key rollover, there should generally only be 2 DNSKEY records. In the CPSC's case, one of the DNSKEYs (id 59364) isn't signing anything. Another of the DNSKEYs (id 27793) is redundant. In other words, the CPSC includes 552 bytes of completely worthless data in every DNS record. Just eliminating these two useless records would cut the maximum size of a DNS Amplification DDoS attack using the CPSC's zone by almost 13 percent.

You can visualize the chain of how DNSSEC records are used to sign each other by using a tool provided by DNSVIZ. The diagram below shows how the two unused DNSKEYs don't actually sign any records.

A Better CPSC Zone

There are a handful of other minor optimizations. Add them all up and the CPSC Zone could be much smaller. At CloudFlare, we've worked to optimize the size and performance of our zone records. We used our automated zone generator on the CPSC and got the following zone file:

cpsc.gov. 281 IN A 104.27.142.66 cpsc.gov. 281 IN A 104.27.143.66 cpsc.gov. 281 IN RRSIG A 13 2 300 20160402121755 20160331101755 35273 cpsc.gov. xYJVr9iNMm50wgoGbMaG76QNyZk2aU2STmzxlUGe09G9LyEqnlDW6YLK pyAeMqHODKuxt83dKDUeB0bgvSPCyQ== cpsc.gov. 278 IN AAAA 2400:cb00:2048:1::681b:8e42 cpsc.gov. 278 IN AAAA 2400:cb00:2048:1::681b:8f42 cpsc.gov. 278 IN RRSIG AAAA 13 2 300 20160402121752 20160331101752 35273 cpsc.gov. LHmzdf6AqsrBVT5ejyoRt2oruXuaHkavhvoNHiqr2OXALnquouOfHsdq qRlHgQ1mkWRbeJ3nwIwDnWbInbIhkg== cpsc.gov. 272 IN MX 5 stagg.cpsc.gov. cpsc.gov. 272 IN MX 5 hormel.cpsc.gov. cpsc.gov. 272 IN RRSIG MX 13 2 300 20160402121746 20160331101746 35273 cpsc.gov. jmAvZk3bMxcqBECvvnhC7Xn2gtdvJnrRqj3xypzQ4Kona3tzs8H5+IBp a9X+TbcIGs1zgIny9UjybVUws+YTqw== cpsc.gov. 86391 IN SOA abby.ns.cloudflare.com. dns.cloudflare.com. 2021076894 10000 2400 604800 3600 cpsc.gov. 86391 IN RRSIG SOA 13 2 86400 20160402121805 20160331101805 35273 cpsc.gov. SrUvxJjlRvbAhFUXlEF6pbilXKLN/gzYcsyi4yYRR5o89k73nMv5mYZr vIi+uaPZe8Qht6nuNWRHhrYxQiUpLw== cpsc.gov. 86367 IN NS lars.ns.cloudflare.com. cpsc.gov. 86367 IN NS abby.ns.cloudflare.com. cpsc.gov. 86367 IN RRSIG NS 13 2 86400 20160402121741 20160331101741 35273 cpsc.gov. QO3FN7XVRMTewKuJGlbS1AglYnQrsbEfiWS65Mp+wT3D6n973Be9XzcR nvel4fPLvQTcIe/h5kpbXjut3nfhFQ== cpsc.gov. 3505 IN DNSKEY 257 3 13 mdsswUyr3DPW132mOi8V9xESWE8jTo0dxCjjnopKl+GqJxpVXckHAeF+ KkxLbxILfDLUT0rAK9iUzy1L53eKGQ== cpsc.gov. 3505 IN DNSKEY 256 3 13 koPbw9wmYZ7ggcjnQ6ayHyhHaDNMYELKTqT+qRGrZpWSccr/lBcrm10Z 1PuQHB3Azhii+sb0PYFkH1ruxLhe5g== cpsc.gov. 3505 IN RRSIG DNSKEY 13 2 3600 20160426183927 20160226183927 2371 cpsc.gov. o0c75Fa7KwogAAEpYIoWuTcsV1fPQ4sfuER+WOtxkOPb0Fe8PdF2yBWa 3gRLA/N2pmdCVrAAeIqdR0NDfX6PCw== cpsc.gov. 47 IN TXT "v=spf1 ip4:63.74.109.6 ip4:63.74.109.10 ip4:63.74.109.20 mx a:list.cpsc.gov -all" cpsc.gov. 47 IN RRSIG TXT 13 2 300 20160402121801 20160331101801 35273 cpsc.gov. LXjZYeejRtkCFOVqqUpupc3I/4s7Ypua/b/xE3wIC+7a4NRjM7+UA90s gDdINQ8ZPF3SLcCqw2nfc3ex2II8Kg== cpsc.gov. 3324 IN NSEC \000.cpsc.gov. A NS SOA WKS HINFO MX TXT AAAA LOC SRV CERT SSHFP IPSECKEY RRSIG NSEC DNSKEY TLSA HIP CDS CDNSKEY OPENPGPKEY SPF cpsc.gov. 3324 IN RRSIG NSEC 13 2 3600 20160402121810 20160331101810 35273 cpsc.gov. 3/1I/SQrwvdffqawg6qG3zfetA1ou5SQioVv0kzMmHbYWJooQv9QEEFX L9RwdWxly0Hly/Aq+UMWzcQ8D2UXZg==

That is 1,389 bytes and close to as optimized as you could manually create. The actual response may be a bit larger because resolvers can add a 320-byte DS record. Even so, the optimized CPSC zone would be about one third the size without compromising any functionality and actually increasing security.

But we can do better than that. Unlike the CPSC's current DNS provider (Verizon/UU.net), CloudFlare has anti-DNS reflection protections in place. Specifically, we automatically upgrade from UDP to TCP when a DNS response is particularly large (generally, over 512 bytes). Since TCP requires a handshake, it prevents source IP address spoofing which is necessary for a DNS amplification attack.

In addition, we rate limit unknown resolvers. Again, this helps ensure that our infrastructure can't be abused to amplify attacks.

Finally, across our DNS infrastructure we have deprecated ANY queries and have proposed to the IETF to restrict ANY queries to only authorized parties. By neutering ANY, we've significantly reduced the maximum size of responses even for zone files that need to be large due to a large number of records.

Denouement

DNSSEC is complicated and it's easy to get wrong. Unfortunately, getting your DNSSEC configuration wrong creates a real potential harm to the rest of the Internet by making your domain's zone file into a potential weapon to be abused by attackers. That's why, in CloudFlare's implementation, we spent a significant amount of time to optimize our automatic DNSSEC configuration to be as efficient as possible while still being easy and free for all our customers.

As Clint Eastwood says in the final scene of the movie "The Good, the Bad, and the Ugly": "You see in this world there's two kinds of people, my friend, those with loaded guns and those who dig. You dig." Pardon the puns but… we hope the CPSC will fix their zone and unload the giant gun attackers have used to launch some of the biggest DDoS attacks on the Internet. When they do, it'll be a relief to "dig" their domain and get back a reasonably sized answer.

If the instructions above aren't enough on their own, we'd be happy to work with the CPSC to help them get their DNS record under control and ensure they are no longer inadvertently facilitating the Internet's largest DDoS attacks. Of course, when the CPSC does get its DNS issues fixed, undoubtedly attackers will find another misconfigured and bloated zone they can abuse to launch DNS amplification DDoS attacks. And, when that happens, we'll be happy to work with them too — until the whole Internet has the well-configured DNS foundation it needs to be safe and secure.

Mediacurrent: QA Building Your Department Part 2: Trust and the Manager-Employee Relationship

Planet Drupal -

About Me

With 15 years of experience in the Information Technology field, and 10 of those years focused on leadership, I’ve learned first hand the value of investing in people and setting them up for success. Before joining Mediacurrent, I started a QA department from scratch and grew it to its current size. Prior to that, I built an IT team. If you're an incoming leader, here's how to start building your own QA/IT department. 

Xeno Media: Posting to Slack, Publishing in Drupal

Planet Drupal -

How Zoomdata employees share insights into company life

Xeno Media is pleased to announce our latest Drupal 7 contrib module, Slack to Drupal.  This module imports pictures uploaded to Slack to Drupal 7 systems--thereby allowing a community of users to add content to a site while managing their daily business collaboration through the Slack app.

Zoomdata--who makes visual analytics software for big data--tasked us with coming up with a solution that allows their employees to submit images for the public website to share the company’s unique, engaging culture to aid in marketing and recruiting.  

Various source platforms, including Instagram, Flickr, and Twitter, were originally considered. As we surveyed Zoomdata employees, though, we realized that Slack was the ideal source. Slack is fundamental to Zoomdata’s work culture; Its 200 employees and contractors throughout North America and Europe actively collaborating on Slack on an ongoing basis. Leveraging Slack as the source platform would allow employees to submit images in real-time without breaking their typical work/collaboration workflows and methods.

With that settled, we started researching how to integrate.  Our developers researched Slack’s API and proposed two approaches: 1) Create a Slack “bot”--a virtual user that our human users could interface with. Or: 2) Integrate with a specific Slack channel.  We elected the later as we could more efficiently access the files in a specific channel and Zoomdata appreciated having a single destination channel for users to come to rather than clogging other channels with off-topic bot chatter.

With the Slack-side figured out, we worked on the Drupal development.  We are supporters of the Drupal Media initiative, and decided to integrate the the Drupal Media 7.x-2.0 File Entity as we do on many of our client sites.  The File Entity module creates an entity like a node for each file in the system.  This allows us to add fields, like Caption, Approval, Date, and Uploader.  It also allows us to use, and reuse the entities in the site on other pieces of content and create views of the entities.  We called this new entity Slack Image.

We also created an administration screen where an administrator can approve or disapprove images.  If images are disapproved, they are removed from the system and not imported again.  If approved, they are available where all the other File Entities are available.

For the Zoomdata public site, we created a view of the new Slack images that appears on their Careers page in a beautiful, modern, and responsive layout using Masonry Views, Colorbox, and GD infinite scroll plugin modules.

Our employees are always posting photos in Slack. I really wanted to share those photos with our customers, partners, prospective employees and vendors so they could get a view inside Zoomdata and know what a great team of people they’re partnering with. Jim, and the team at Xeno Media, made it possible by creating a fantastic Drupal website for us, and by developing Slack to Drupal.

Robyn Forman, Zoomdata’s VP of Digital Marketing.

Results so far have been very positive--with more than half of the company joining the channel and submissions coming from every office and department.  Through Slack to Drupal, employees from throughout the organization have shown what an engaged, fun, and cutting edge culture Zoomdata really is.

Roy Scholten: New process, new results

Planet Drupal -

We’re probably misusing the term MVP when we try to frame what we would like to see make it into core. But the actual mode of working we use there is quite an achievement. We used to grind it out endlessly, where proposed changes could be discussed endlessly, with a high risk of not committing anything at all in the end. What we’re doing now is: agree up front that it’s a good idea to improve feature X or rework interface Y. And then focus on keeping the scope as small as possible.

Yes, I, J and K are also good ideas, but we’re trying to do X here and while these are all related ideas and together would like make for a nicer whole, we should really focus on shipping X, and X alone, before turning our attention to I, J and K. If at all, because while shiny, interface Y actually presents people with more problems, so maybe we should focus on that. Though it’s never that strongly a case of either/or, and we should definately not stop iterating after the initial commit.

This is a very new and different way of working. Deliberately lowering our standards for the goal of introducing change. This is uncomfortable at times, but even that is good, because it means we’re stretching ourselves, which means we’re doing and learning new things. I’m excited and proud to see this happen. More like this.

Doing it like this means that Drupal 8.2:

  • Has content moderation tools (draft! review! publish! etc.)
  • Provides a new way to add new elements (blocks) to the page you’re on, without having to go to some far away corner in the admin section
  • Those elements (blocks! menus! logo & site name! etc.) can then also be configured in the context of the user facing page. A side tray will show up and expose the relevant settings.

Looking forward to learn how these additions will be received and how we can improve them. In the mean time, lets add more useful and usable things to 8.3 (sample content! media handling! better dates! etc).

Tags: drupaluxdrupalplanetSub title: This is a pretty radical change

DrupalCon News: An Insider's Guide to Visiting Dublin

Planet Drupal -

Thinking of coming to DrupalCon Dublin this year? Why not extend your trip by a few days and stay a bit longer to take in some of the fabulous things you can go do and see in Dublin?

Here's our recommended list of things to do and see while here:

1. Guinness Storehouse

Drupal core announcements: Coding standards ratified changes and ongoing proposals

Planet Drupal -

The TWG coding standards committee is announcing two coding standards changes for final discussion. These appear to have reached a point close enough to consensus for final completion. The new process for proposing and ratifying changes is documented on the coding standards project page.

Official coding standards updates now ratified:

Issues awaiting core approval:

Issues that just need a little TLC (you can help!):

These proposals will be re-evaluated during the next coding standards meeting currently scheduled for August 30th. At that point the discussion may be extended, or if clear consensus has been reached one or more policies may be dismissed or ratified and moved to the next step in the process.

FFW Agency: The ABC's of Drupal: Dev Ops, Display and Distribution

Planet Drupal -

The ABC's of Drupal: Dev Ops, Display and Distribution Ray Saltini Wed, 08/24/2016 - 18:43

For anyone who's ever looked up a definition of a Drupal term and been left wondering what it all means, here are some practical real world explanations you can use to navigate the Drupalverse. Watch this space and use comments to send us your feedback and requests.

The Discipline of Dev Ops

Dev Ops, or Development Operations, is the intersection between IT managed hosting support and development. While it is a specialization in many organizations, senior developers, tech leads, and architects should be conversant in the various systems and tools to be used by your IT team or provider.

One of the primary goals of Dev Ops is to create standardized operating system iterations that are consistently reliable and easily replicable. Your unique infrastructure or hosting service plays a big role in these systems, which is why they tend to be customized to each project.

Standardized Dev Ops systems are used to create local and remote development environments, as well as staging and production environments, which all function in the same way. Having good Dev Ops systems in place means that your organization can support continuous development practices like version control and automated testing.

For any site that’s even moderately complex, having Dev Ops standards is huge. You don’t have to try to become a Dev Ops genius yourself: instead, you can find an organization like FFW to provide the level of Dev Ops help and support that is appropriate for the size and scope of your project.

Defining a Display

Displays, unlike Dev Ops, are a little simpler. A Display in Drupal typically refers to how queried data is organized and shown to visitors. It is usually used in connection with a native database query referred to as a View.

One View (or database query) can have several Displays sorted in different ways. For instance, a set of queried data can be output in the following ways:

  • a sortable table
  • a grid
  • as consecutive field sets
  • in a rotating banner
  • as a calendar or list of coming events
  • as points on a map

… and these are only just a few examples of the many different kinds of Displays.

The Details Around Distributions

A Distribution is a pre-developed assembly of database data, code, and files. Distributions commonly include saved content, configuration settings, Drupal core, contributed and custom modules, libraries, and a custom theme. It’s basically a pre-built Drupal site.

Most people first become acquainted with Distributions as different iterations of Drupal that are built for specific use cases or verticals, such as e-commerce or publishing. Many distributions are robust, production-ready applications that will save you tremendous work. They let you take advantage of the distribution sponsor’s subject matter expertise.

There are other kinds of distributions, such as ones developed mainly for marketing purposes to showcase what Drupal can do and how Drupal can be used. Both of these types of distributions have value, but it is important to differentiate between the two.

Distributions can be vetted in much the same way that a Drupal module or theme can be vetted. When evaluating a Distribution, I always like to ask the following questions:

  • Who are the contributors?
  • What is their experience?
  • Is the project actively maintained and are new features or versions planned?

The other primary consideration when vetting a Distribution is how much complexity and effort is required to ‘unravel’ a distribution. Many organizations have found that the more fully realized distributions are difficult to customize around their specific workflows and therefore are more expensive to change than starting fresh with a more basic version of Drupal.

If you want to know more about Distributions, I recommend looking at Drupal’s distribution project pages and this documentation page.

Tagged with Comments

Drupal.org blog: Upcoming Changes to the Front Page

Planet Drupal -

In recent weeks we've been making several small changes to Drupal.org: precursors to bigger things to come. First, we moved the user activity links to a user menu in the header. Next, we're moving the search function from the header to the top navigation. These changes aren't just to recover precious pixels so you can better enjoy those extra long issue summaries—these are the first step towards a new front page on Drupal.org.

As the Drupal 8 life-cycle has moved from development, to release, to adoption, we have adapted Drupal.org to support the needs of the project in the moment. And today, the need of the moment is to support the adoption journey.

As we make these changes you'll see echoes of the visual style we used when promoting the release of Drupal 8.

  • The Drupal wordmark region will help to define Drupal, and promote trying a demo.

  • A ribbon will promote contextual CTAs like learning more about Drupal 8.

  • The news feed will be tweaked.

  • DrupalCon will have a permanent home on the front page.

  • Community stats and featured case studies will be carried over(but may evolve).

  • The home page sponsorship format may change.

  • We'll be phasing in a new font throughout the site: Ubuntu - which you've already seen featured in the new Documentation section.

Here's a teaser

… a sneak preview of some new page elements and styles you'll see in the new home page.  

Our first deployment will introduce the new layout and styles. Additional changes will follow as we introduce content to support our turn towards the adoption journey. Drupal evaluators beginning their adoption journey want to know who uses Drupal, and what business needs Drupal can solve. We will begin promoting specific success stories: solutions built in Drupal to meet a concrete need.

What's next?

We're continuing to refine our content model and editorial workflow for the new front page. You'll see updates in the Drupal.org change notifications as we get closer to deployment.

Wondering why we're making these changes now? This turn towards the adoption journey is part of our changing priorities for the next 12 months.

Upcoming Changes to the Front Page

Drupal News -

In recent weeks we've been making several small changes to Drupal.org: precursors to bigger things to come. First, we moved the user activity links to a user menu in the header. Next, we're moving the search function from the header to the top navigation. These changes aren't just to recover precious pixels so you can better enjoy those extra long issue summaries—these are the first step towards a new front page on Drupal.org.

As the Drupal 8 life-cycle has moved from development, to release, to adoption, we have adapted Drupal.org to support the needs of the project in the moment. And today, the need of the moment is to support the adoption journey.

As we make these changes you'll see echoes of the visual style we used when promoting the release of Drupal 8.

  • The Drupal wordmark region will help to define Drupal, and promote trying a demo.

  • A ribbon will promote contextual CTAs like learning more about Drupal 8.

  • The news feed will be tweaked.

  • DrupalCon will have a permanent home on the front page.

  • Community stats and featured case studies will be carried over(but may evolve).

  • The home page sponsorship format may change.

  • We'll be phasing in a new font throughout the site: Ubuntu - which you've already seen featured in the new Documentation section.

Here's a teaser

… a sneak preview of some new page elements and styles you'll see in the new home page.  

Our first deployment will introduce the new layout and styles. Additional changes will follow as we introduce content to support our turn towards the adoption journey. Drupal evaluators beginning their adoption journey want to know who uses Drupal, and what business needs Drupal can solve. We will begin promoting specific success stories: solutions built in Drupal to meet a concrete need.

What's next?

We're continuing to refine our content model and editorial workflow for the new front page. You'll see updates in the Drupal.org change notifications as we get closer to deployment.

Wondering why we're making these changes now? This turn towards the adoption journey is part of our changing priorities for the next 12 months.

TONIGHT: August Twin Cities Drupal user group and Lab Hours

Twin Cities Drupal Group -

Start:  2016-08-24 18:00 - 21:00 America/Chicago Organizers:  jerdavis Barry Madore Event type:  User group meeting

The fourth Wednesday of the month is, well, tonight!! And you know what that means.

This month's topic is Show & Tell & Bring a Question. Show off a shiny new site you've just launched, bring a project you're working on, bring a question you've been dying to ask. And if there's time, we'll follow up with a demo of the results we've seen after taking the advice the group provided last month to one questioner. Refreshing your memory, the group advised that the questioner create a custom Migrate script to pull in images from a CDN into their Drupal site. Well, it's working!

Lab Hours are available tonight as well, but starting at 6 PM, one hour later than usual.

Signup below or comment to let Jer know you're coming!


TCDUG and Community Lab Hours meetups are hosted by Advantage Labs and take place in their space at Intermedia Arts. Special thanks to Allie and Jer, who have provided pizza and beverages for this group since the beginning! That's a LOT of beer and pizza!

Intermedia Arts
2822 Lyndale Ave S
Minneapolis, MN 55408
map & directions

This is near the corner of Lake and Lyndale. There is parking in the lot to the right of the building (north side).

~Don't forget to signup and let us know you are coming!

Chocolate Lily: Announcing Drutopia

Planet Drupal -

Drutopia is an initiative within the Drupal project that prioritizes putting the best online tools into the hands of grassroots groups. By embracing the liberatory possibilities of free software and supporting people-centred economic models, Drutopia aims to revolutionize the way we work and cooperate.

Drutopia is at once an ethos of Drupal development and a fresh take on Drupal distributions for users to build upon, all based in a governance model that gives users a large role in the direction of the project.

Core values of the Drutopia initiative include:

  • Be inclusive regarding gender, gender identity, sexual orientation, ethnicity, ability, age, religion, geography and class.
  • Commit to protection of personal information and privacy and freedom from surveillance.
  • Put collaboration and cooperation above competition.
  • Prioritize human needs over private profit.
  • Foster non-hierarchical structures and collective decision-making.

Drutopia focuses on shared solutions. Drupal excels at providing the tools to develop and distribute specialized website platforms that can be freely shared, reused, and adapted. Of the three most-used free software content management systems (CMSs) – WordPress, Joomla!, and Drupal – only Drupal has the built-in ability to package and share highly developed distributions.

Distributions are essential in attracting and meeting the needs of groups that want to support the free software movement but don’t have the technical know-how or resources to create a site from scratch. For developers, too, distributions hold a lot of potential because they do the heavy lifting of initial setup, allowing developers and site builders to bypass many hours of unnecessary effort. Drupal distributions so far have been held back by a series of factors that Drutopia aims to address.

Drutopia is about returning to Drupal’s roots in free software and progressive social change. Since its founding years, the Drupal free software project has both reflected and contributed to the democratic potential of the internet: to empower citizens to freely collaborate and organize outside the control of governments and corporate media. Long before it powered Fortune 500 sites and whitehouse.gov, Drupal was a tool of choice for small, grassroots, change-oriented groups.

This initiative aims to reclaim Drupal for the communities and groups that have always been its core users and adopters and have contributed to much of its best innovation.

Join us at drutopia.org.

Frederick Giasson: Winnipeg City’s NOW [Data] Portal

Planet Drupal -

The Winnipeg City’s NOW (Neighbourhoods Of Winnipeg) Portal is an initiative to create a complete neighbourhood web portal for its citizens. At the core of the project we have a set of about 47 fully linked, integrated and structured datasets of things of interests to Winnipegers. The focal point of the portal is Winnipeg’s 236 neighbourhoods, which define the main structure of the portal. The portal has six main sections: topics of interests, maps, history, census, images and economic development. The portal is meant to be used by citizens to find things of interest in their neibourhood, to learn their history, to see the images of the things of interest, to find tools to help economic development, etc.

The NOW portal is not new; Structured Dynamics was also its main technical contractor for its first release in 2013. However we just finished to help Winnipeg City’s NOW team to migrate their older NOW portal from OSF 1.x to OSF 3.x and from Drupal 6 to Drupal 7; we also trained them on the new system. Major improvements accompany this upgrade, but the user interface design is essentially the same.

The first thing I will do is to introduce each major section of the portal and I will explain the main features of each. Then I will discuss the new improvements of the portal.

Datasets

A NOW portal user won’t notice any of this, but the main feature of the portal is the data it uses. The portal manages 47 datasets (and growing) of fully structured, integrated and linked datasets of things of interests to Winnipegers. What the portal does is to manage entities. Each kind of entity (swimming pools, parks, places, images, addresses, streets, etc.) are defined with multiple properties and values. Several of the entities reference other entities in other datasets (for example, an assessment parcel from the Assessment Parcels dataset references neighbourhoods entities and property addresses entities from their respective datasets).

The fact that these datasets are fully structured and integrated means that we can leverage these characteristics to create a powerful search experience by enabling filtering of the information on any of the properties, to bias the searches depending where a keyword search match occurs, etc.

Here is the list of all the 47 datasets that currently exists in the portal:

  1. Aboriginal Service Providers
  2. Arenas
  3. Neighbourhoods of Winnipeg City
  4. Streets
  5. Economic Development Images
  6. Recreation & Leisure Images
  7. Neighbourhoods Images
  8. Volunteer Images
  9. Library Images
  10. Parks Images
  11. Census 2006
  12. Census 2001
  13. Winnipeg Internal Websites
  14. Winnipeg External Websites
  15. Heritage Buildings and Resources
  16. NOW Local Content Dataset
  17. Outdoor Swimming Pools
  18. Zoning Parcels
  19. School Divisions
  20. Property Addresses
  21. Wading Pools
  22. Electoral wards of Winnipeg City
  23. Assessment Parcels
  24. Libraries
  25. Community Centres
  26. Police Service Centers
  27. Community Gardens
  28. Leisure Centres
  29. Parks and Open Spaces
  30. Community Committee
  31. Commercial real estates
  32. Sports and Recreation Facilities
  33. Community Characterization Areas
  34. Indoor Swimming Pools
  35. Neighbourhood Clusters
  36. Fire and Paramedic Stations
  37. Bus Stops
  38. Fire and Paramedic Service Images
  39. Animal Services Images
  40. Skateboard Parks
  41. Daycare Nurseries
  42. Indoor Soccer Fields
  43. Schools
  44. Truck Routes
  45. Fire Stations
  46. Paramedic Stations
  47. Spray Parks Pads
Structured Search

The most useful feature of the portal to me is its full-text search engine. It is simple, clean and quite effective. The search engine is configured to try to give the most relevant results a NOW portal user may be searching. For example, it will positively bias some results that comes from some specific datasets, or matches that occurs in specific property values. The goal of this biasing is to improve the quality of the returned results. This is somewhat easy to do since the context of the portal is well known and we can easily boost scoring of search results since everything is fully structured.

Another major gain is that all the search results are fully templated. The search results do not simply return a title and some description for your search results. It does template all the information the system has about the matched results, but also displays the most relevant information to the users in the search results.

For example, if I search for a indoor swimming pool, in most of the cases it may be to call the front desk to get some information about the pool. This is why different key information will be displayed directly in the search results. That way, most of the users won’t even have to click on the result to get the information they were looking for directly in the search results page.

Here is an example of a search for the keywords main street. As you can notice, you are getting different kind of results. Each result is templated to get the core information about these entities. You have the possibility to focus on particular kind of entities, or to filter by their location in specific neighbourhoods.

Templated Search Results

Now let’s see some of the kind of entities that can be searched on the portal and how they are presented to the users.

Here is an example of an assessment parcel that is located in the St. John’s neighbourhood. The address, the value, the type and the location of the parcel on a map is displayed directly into the search results.

Another kind of entity that can be searched are the property addresses. These are located on a map, the value of the parcels and the building and the zoning of the address is displayed. The property is also linked to its assessment parcel entity which can be clicked to get additional information about the parcel.

Another interesting type of entity that can be searched are the streets. What is interesting in this case is that you get the complete outline of the street directly on a map. That way you know where it starts and where it ends and where it is located in the city.

There are more than a thousand geo-localized images of all different things in the city that can be searched. A thumbnail of the image and the location of the thing that appears on the image appears in the search results.

If you were searching for a nursery for your new born child, then you can quickly see the name, location on a map and the phone number of the nursery directly in the search result.

There are just a few examples of the fifty different kind of entities that can appear like this in the search results.

Mapping

The mapping tool is another powerful feature of the portal. You can search like if you were using the full-text search engine (the top search box on the portal) however you will only get the results that can be geo-localized on a map. You can also simply browse entities from a dataset or you can filter entities by their properties/values. You can persist entities you find on the map and save the map for future reference.

In the example below, it shows that someone searched for a street (main street) and then he persisted it on the map. Then he search for other things like nurseries and selected the ones that are near the street he persisted, etc. That way he can visualize the different known entities in the portal on a map to better understand where things are located in the city, what exists near a certain location, within a neighbourhood, etc.

Census Analysis

Census information is vital to the good development of a city. They are necessary to understand the trends of a sector, who populates it, etc., such that the city and other organizations may properly plan their projects to have has much impact as possible.

These are some of the reason why one of the main section of the site is dedicated to census data. Key census indicators have been configured in the portal. Then users can select different kind of regions (neighbourhood clusters, community areas and electoral wards) to get the numbers for each of these indicators. Then they can select multiple of these regions to compare each other. A chart view and a table view is available for presenting the census data.

History, Images & Points of Interest

The City took the time to write the history of each of its neighbourhoods. In additional to that, they hired professional photographs to photograph the points of interests of the city, to geo-localize them and to write a description for each of these photos. Because of this dedication, users of the portal can learn a much about the city in general and the neighbourhood they live in. This is what the History and Image sections of the website are about.

Historic buildings are displayed on a map and they can be browsed from there.

Images of points of interests in the neighbourhood are also located on a map.

Find Your Neighbourhood

Ever wondered in which neighbourhood you live in? No problem, go on the home page, put your address in the Find your Neighbourhood section and you will know it right away. From there you can learn more about your neighbourhood like its history, the points of interest, etc.

Your address will be located on a map, and your neighbourhood will be outlined around it. Not only you will know in which neighbourhood you live, but you will also know where you live within it. From there you can click on the name of the neigbourhood to get to the neighbourhood’s page and start learning more about it like its history, to see photos of points of interest that exists in your neighbourhood, etc.

Browsing Content by Topic

Because all the content of the portal is fully structured, it is easy to browse its content using a well defined topic structure. The city developed its own ontology that is used to help the users browse the content of the portal by browsing topics of interest. In the example below, I clicked the Economic Development node and then the Land use topic. Finally I clicked the Map button to display things that are related to land use: in this case, zoning and assessment parcels are displayed to the user.

This is another way to find meaningful and interesting content from the portal.

Depending on the topic you choose, and the kind of information related to that topic, you may end up with different options like a map, a list of links to documents related to that topic, etc.

Export Content

Now that I made an overview of each of the main features of the portal, let’s go back to the geeky things. The first thing I said about this portal is that at its core, all information it manages is fully structured, integrated and linked data. If you get to the page of an entity, you have the possibility to see the underlying data that exists about it in the system. You simply have to click the Export tab at the top of the entity’s page. Then you will have access to the description of that entity in multiple different formats.

In the future, the City should (or at least I hope will) make the whole set of datasets fully downloadable. Right now you only have access to that information via that export feature per entity. I hope because this NOW portal is fully disconnected from another initiative by the city: data.winnipeg.ca, which uses Socrata. The problem is that barely any of the datasets from NOW are available on data.winnipeg.ca, and the ones that are appearing are the raw ones (semi-structured, un-documented, un-integrated and non-linked) all the normalization work, the integration work, the linkage work done by the NOW team hasn’t been leveraged to really improve the data.winnipeg.ca datasets catalog.

New with the upgrades

Those who are familiar with the NOW portal will notice a few changes. The user interface did not change that much, but multiple little things got improved in the process. I will cover the most notable of these changes.

The major changes that happened are in the backend of the portal. The data management in OSF for Drupal 7 is incompatible with what was available in Drupal 6. The management of the entities became easier, the configuration of OSF networks became a breeze. A revisioning system has been added, the user interface is more intuitive, etc. There is no comparison possible. However, portal users’ won’t notice any of this, since these are all site administrator functions.

The first thing that users will notice is the completely new full-text search engine. The underlying search engine is almost the same, but the presentation is far better. All entity types have gotten their own special template, which are displayed in a special way in the search results. Most of the time results should be much more relevant, filtering is easier and cleaner. The search experience is much better in my view.

The overall site performance is much better since different caching strategies have been put in place in OSF 3.x and OSF for Drupal. This means that most of the features of the portal should react more swiftly.

Now every type of entity managed by the portal is templated: their webpage is templated in specific ways to optimize the information they want to convey to users along with their search result “mini page” when they get returned as the result of a search query.

Multi-linguality is now fully supported by the portal, however not everything is currently templated. However expect a fully translated NOW portal in French in the future.

Creating a Network of Portals

One of the most interesting features that goes with this upgrade is that the NOW portal is now in a position to participate into a network of OSF instances. What does that mean? Well, it means that the NOW portal could create partnerships with other local (regional, national or international) organizations to share datasets (and their maintenance costs).

Are there other organizations that uses this kind of system? Well, there is at least another one right in Winnipeg City: MyPeg.ca, also developed by Structured Dynamics. MyPeg uses RDF to model its information and uses OSF to manage its information. MyPeg is a non-profit organization that uses census (and other indicator) data to do studies on the well being of Winnipegers. The team behind MyPeg.ca are research experts in indicator data. Their indicator datasets (which includes census data) is top notch.

Let’s hypothetize that there would be interest between the two groups to start collaborating. Let’s say that the NOW portal would like to use MyPeg’s census datasets instead of its own since they are more complete, accurate and include a larger number of important indicators. What they basically want is to outsource the creation and maintenance of the census/indicators data to a local, dedicated and highly professional organization. The only things they would need to do is to:

  1. Formalize their relationship by signing a usage agreement
  2. The NOW portal would need to configure the MyPeg.ca OSF network into their OSF for Drupal instance
  3. The NOW portal would need to register the datasets it want to use from MyPeg.ca.

Once these 3 steps are done, taking no more than a couple of minutes, then the system administrators of the NOW portal could start using the MyPeg.ca indicator datasets like they were existing on their own network. (The reverse could also be true for MyPeg.) Everything would be transparent to them. From then on, all the fixes and updates performed by MyPeg.ca to their indicator datasets would immediately appear on the NOW portal and accessible to its users.

This is one possibility to collaborate. Another possibility would be to simply on a routine basis (every month, every 6 months, every year) share the serialized datasets such that the NOW portal re-import the dataset from the files shared by MyPeg.ca. This is also possible since both organizations use the same Ontology to describe the indicator data. This means that no modification is required by the City to take that new information into account, they only have to import and update their local datasets. This is the beauty of ontologies.

Conclusion

The new NOW portal is a great service for citizens of Winnipeg City. It is also a really good example of a web portal that leverages fully structured, integrated and linked data. To me, the NOW portal is a really good example of the features that should go along with a municipal data portal.

Pages

Subscribe to Cruiskeen Consulting LLC aggregator