General Raspberry Pi stuff

Subjects that don't have their own home
Post Reply
User avatar
jack
Thermionic Monk Status
Posts: 5493
Joined: Wed Dec 29, 2010 8:58 pm
Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
Contact:

#61 Re: General Raspberry Pi stuff

Post by jack »

Got Pi-hole and the UniFi controller installed in docker with a stack YAML definition.

Also installed Portainer.io which is great for managing a docker environment along with Node-RED which is just hoopy.

Time to climb the wooden stairs to Bedfordshire...

IMG_20200414_222830_copy_1209x907.jpg
Vivitur ingenio, caetera mortis erunt
Neal
Shed dweller
Posts: 2299
Joined: Fri Aug 03, 2007 10:57 am
Location: From the land of the Bodgers

#62 Re: General Raspberry Pi stuff

Post by Neal »

Neat, never crossed my mind to use docker! I’m going to have a go at this :bounce:
Only the Sith deal in absolutes.
User avatar
jack
Thermionic Monk Status
Posts: 5493
Joined: Wed Dec 29, 2010 8:58 pm
Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
Contact:

#63 Re: General Raspberry Pi stuff

Post by jack »

Neal wrote: Wed Apr 15, 2020 12:01 am Neat, never crossed my mind to use docker! I’m going to have a go at this :bounce:
Pi-hole, Portainer.io and the UniFi Controller are all available as official supported builds within the docker hub, i.e. they are always the latest version and docker will automatically do the download and cache the latest version for you. It's really clean.

I'll post the YAML definition file for this setup should anyone want to have a go.

One last thing - all the data used by an application in a container is also normally in that container, meaning that you have to backup the container to backup the data.

This is obviously inefficient, so I've mapped the data from each application to be outside the container in a set of subdirectories, i.e under ~pi/volumes/[appname]/... This is also in the YAML stack definition file.

Now setting up a cron job to auto backup this tree to Google drive or Dropbox every day. All applications are therefore backed up in one easy process... As long as you have the YAML and the data, restoring a system is trivial.

Next bit is moving the media server onto it too...
Vivitur ingenio, caetera mortis erunt
User avatar
jack
Thermionic Monk Status
Posts: 5493
Joined: Wed Dec 29, 2010 8:58 pm
Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
Contact:

#64 Re: General Raspberry Pi stuff

Post by jack »

Reading the Pi 4 boot ROM release notes at https://github.com/raspberrypi/rpi-eepr ... e-notes.md

...I noticed the following:
on the 18/11/2019, the RPi 4 boot ROM release notes wrote:First release candidate before this beta is moved to a stable release series.
  • Avoid resetting TFTP prefix after retries or if start4.elf is not found.
  • Add MAC_ADDRESS option which allows the OTP Ethernet MAC address to be
    overriden. An VideoCore firmware update will propagate this forced
    mac address to device-tree/cmdline in the near future.
  • Various internal refactorings to prepare for USB MSD storage boot in
    the next beta-series.
  • Enable high-speed mode for EMMC cards.
...so there is hope for the near future :)

Network boot was added on 23/09/19 and they said that USB MSD boot would follow that, a new Beta ROM was started on 09/04/20 plus ROM updates seem to happen every few days for the Pi 4, so fingers crossed!

Edit: Have ordered my first 4GB Pi 4 from Farnell and some extras (fun stuff) for it from Banggood & AliExpress.

I've ordered the 4GB as I think that the 2GB might be a bit tight on memory for all I have planned for this machine... The 2GB Pi 4 is a LOT cheaper (unreasonably so, IMHO), so I may end up using a 2GB one for the networking & media server stuff and the 4GB one in the workshop...

Whatever ends up happening, I'm feeling that the 1GB RAM of the Pi 3B/B+ is VERY limiting for this number of apps with docker...
Vivitur ingenio, caetera mortis erunt
SimonC
Old Hand
Posts: 545
Joined: Mon Sep 24, 2007 11:34 am
Location: West Yorkshire

#65 Re: General Raspberry Pi stuff

Post by SimonC »

When you reach the limits of the Pi4 then Docker swarm on this is maybe the next step? https://turingpi.com/

I've been lurking on this thread and eyeing up my box of Pi3's in the corner for a lock-down project. Pi-hole looks to be a good investment for one of them, along with teaching myself python and cracking on with a bit of home automation...
Neal
Shed dweller
Posts: 2299
Joined: Fri Aug 03, 2007 10:57 am
Location: From the land of the Bodgers

#66 Re: General Raspberry Pi stuff

Post by Neal »

Interesting but having worked with hyper converged systems before, I’m not sure a 100Mbps communication bus between nodes really cuts it! Struggling to see the problem that unit is trying fix.
Only the Sith deal in absolutes.
User avatar
jack
Thermionic Monk Status
Posts: 5493
Joined: Wed Dec 29, 2010 8:58 pm
Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
Contact:

#67 Re: General Raspberry Pi stuff

Post by jack »

I've been playing with docker swarm too - it's desperately cool...

...however CPU power and threads are not the limiting factors in this project - I'm not mining bitcoin or doing prime factoring!

Memory is the issue - the 3 series are only 1GB max. I want simplicity, i.e. a single host if I can get away with that. Plus the Pi 4 has proper 1GB ethernet (i.e. not just hanging off the USB controller), and the USB controller is genuine 3.0, so the M.2 SATA SSD is even faster (not that that is an issue - the SSD is about reliability here, not speed).

Pi-hole, in a domestic environment, is actually pretty economic with resources, even with the 40+ devices we have - it's amazing how many bits of odd kit crop up when you have a decent managed network that's capable of seeing what's out there... every MAC address has been tracked and identified.

Kubernetes, IMHO, in a domestic environment is overkill, however docker itself has a small footprint - definitely a lightweight shim and Portfolio.io does an excellent job of managing it.

The main resource hog at the moment is the UniFi Controller. Each on their own on a Pi 3B is fine, but together the memory gets tight, and then I still need to add a media server, plus whatever gets added in the future (and there will be stuff).

So, keeping it simple and just spending the extra 20 quid or so now seems a more sensible approach.

Bizarrely, using docker really does simplify a lot of stuff. The more I learn about this ecosystem, the more I like it.
Vivitur ingenio, caetera mortis erunt
User avatar
jack
Thermionic Monk Status
Posts: 5493
Joined: Wed Dec 29, 2010 8:58 pm
Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
Contact:

#68 Re: General Raspberry Pi stuff

Post by jack »

Pi 4 has arrived from Farnell, seen here with nice heatsinks etc. from Banggood. Also POE hat from Farnell plus other bits...

Small issue with the heatsink in that the rubber sticky pads they provided for thermal coupling were actually 3M double sided foam tape - an INSULATOR! https://www.3m.co.uk/3M/en_GB/company-u ... Tape-9448A

Replaced with the correct stuff - a bit of this: https://uk.farnell.com/bergquist/gp5000 ... dp/1893450 - just as well I checked...

Onwards!
IMG_20200421_175451_copy_756x1008.jpg
Vivitur ingenio, caetera mortis erunt
User avatar
jack
Thermionic Monk Status
Posts: 5493
Joined: Wed Dec 29, 2010 8:58 pm
Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
Contact:

#69 Re: General Raspberry Pi stuff

Post by jack »

New office server up and running. Really like the Pi 4, but I'd forgotten to order the microHDMI->HDMI adapters, so had to wait for some from Amazon...

Note I'm using a standard Anker USB C charger brick (which does proper USB C), rather than the normal Pi 4 charger...
IMG_20200425_131844_copy_756x1008.jpg
Vivitur ingenio, caetera mortis erunt
User avatar
ed
retired
Posts: 5384
Joined: Thu Jun 21, 2007 4:01 pm
Location: yorkshire
Contact:

#70 Re: General Raspberry Pi stuff

Post by ed »

wicked!

just finished configuring the pi 4b with lamp. Ported all the test databases and php stuff and it all runs as well after some spanner work under the bonnet. I wasn't at all surprised because there where serious mismatches between the advertised latest versions of mysql and phpmyadmin but it's now purring.

Response seems much sharper than the old windows machine running wamp.

I can now retire the homeplug from the windows machine and dispose of wamp.

there is probably a lot more the pi 4 can do as I don't think php, mysql and html testing is going to trouble it much. I wonder if there is any benefit taking the LMS off the pi 3 and just leaving it with squeezelite.......
There's nowhere you can be that isn't where you're meant to be
User avatar
jack
Thermionic Monk Status
Posts: 5493
Joined: Wed Dec 29, 2010 8:58 pm
Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
Contact:

#71 Re: General Raspberry Pi stuff

Post by jack »

ed wrote: Fri May 01, 2020 6:16 pm there is probably a lot more the pi 4 can do ...
Indeed. How much RAM on your Pi 4?

Put docker & Portainer.io on it, then run LAMP in a container.

Then you can run LMS in another container...

...and Pi-hole in another...

...and so on...

FWIW, I'll have a docker/Portainer stack with all the setup for that in a day or so...
Vivitur ingenio, caetera mortis erunt
User avatar
ed
retired
Posts: 5384
Joined: Thu Jun 21, 2007 4:01 pm
Location: yorkshire
Contact:

#72 Re: General Raspberry Pi stuff

Post by ed »

4gb

I'm getting more and more impressed with it as an apache server.

my only gripe so far is that the red led at the front is too bright.

all the docker stuff is a bit ott for me at the moment but if I expand it's workload in the future I'll take another look.
There's nowhere you can be that isn't where you're meant to be
User avatar
jack
Thermionic Monk Status
Posts: 5493
Joined: Wed Dec 29, 2010 8:58 pm
Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
Contact:

#73 Re: General Raspberry Pi stuff

Post by jack »

That sounds great Ed...

One small thing to think about.

Normal uSD cards are designed for the occasional photo, not for extensive writing. If they do wear-levelling at all, it's pretty crude. Also the cell technology is not designed for high write counts.

I've had even good quality uSD cards (e.g. SanDisk) fail after a couple of months, hence my move to SSDs for real work.

However, you can get "industrial" uSD cards and cards designed for video recording - these are more expensive but use better quality cells, more standby (spare) cells and better wear levelling. It may be worth investing in a more appropriate card. Typically rated between 25x to 200x the write tolerance of standard uSD cards.

A lamp stack will write logs etc. as well as data from your apps, so I'd really be sure to have regular backups of your system...
Last edited by jack on Mon May 04, 2020 9:31 am, edited 1 time in total.
Vivitur ingenio, caetera mortis erunt
User avatar
ed
retired
Posts: 5384
Joined: Thu Jun 21, 2007 4:01 pm
Location: yorkshire
Contact:

#74 Re: General Raspberry Pi stuff

Post by ed »

thanks for the tips Nick.

It maybe that I've chanced upon a good sd card by accident. I've just done a bit of googling and the one I used was a Sandisk ultra 64gb which appears to have wear levelling. It also seems to be quite fast at 100MB/s. This is all new to me, I only went for the largest and cheapest card card at Ebuyer.

backup? what's a backup! snigger
There's nowhere you can be that isn't where you're meant to be
User avatar
jack
Thermionic Monk Status
Posts: 5493
Joined: Wed Dec 29, 2010 8:58 pm
Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
Contact:

#75 Re: General Raspberry Pi stuff

Post by jack »

ed wrote: Mon May 04, 2020 9:30 am ...Sandisk ultra...
These are the same ones I use as my general uSD card for the Pis - all my audio clients have them... and they fail every few months. I have 7 Pis doing stuff in the house - one uSD card fails every couple of months...

SanDisk do indeed have wear-levelling, but it's the memory cell technology of the card which ultimately determines the number of safe write cycles.

The speed of the card is not that important for Pis - it's really about saving and reading big photos to/from the card quickly. For general Pi use, a standard C10 card is more than adequate. It's also worth noting that the Pi 4's uSD interface is twice as fast as the 3B+; likewise the uSD is no longer a shared controller and is genuine USB 3 (on the two blue ports). A USB-connect SSD is at least an order of magnitude faster than a uSD card.

Should also point out that there's no standard for wear-levelling - each manufacturer can choose to do their own thing (or nothing at all) - so in a given price bracket/use case some are better than others.

However, for reliability, go for the industrial or video/security camera specified ones.
Vivitur ingenio, caetera mortis erunt
Post Reply