General Raspberry Pi stuff
- jack
- Thermionic Monk Status
- Posts: 5493
- Joined: Wed Dec 29, 2010 8:58 pm
- Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
- Contact:
#61 Re: General Raspberry Pi stuff
Got Pi-hole and the UniFi controller installed in docker with a stack YAML definition.
Also installed Portainer.io which is great for managing a docker environment along with Node-RED which is just hoopy.
Time to climb the wooden stairs to Bedfordshire...
Also installed Portainer.io which is great for managing a docker environment along with Node-RED which is just hoopy.
Time to climb the wooden stairs to Bedfordshire...
Vivitur ingenio, caetera mortis erunt
-
- Shed dweller
- Posts: 2299
- Joined: Fri Aug 03, 2007 10:57 am
- Location: From the land of the Bodgers
#62 Re: General Raspberry Pi stuff
Neat, never crossed my mind to use docker! I’m going to have a go at this
Only the Sith deal in absolutes.
- jack
- Thermionic Monk Status
- Posts: 5493
- Joined: Wed Dec 29, 2010 8:58 pm
- Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
- Contact:
#63 Re: General Raspberry Pi stuff
Pi-hole, Portainer.io and the UniFi Controller are all available as official supported builds within the docker hub, i.e. they are always the latest version and docker will automatically do the download and cache the latest version for you. It's really clean.
I'll post the YAML definition file for this setup should anyone want to have a go.
One last thing - all the data used by an application in a container is also normally in that container, meaning that you have to backup the container to backup the data.
This is obviously inefficient, so I've mapped the data from each application to be outside the container in a set of subdirectories, i.e under ~pi/volumes/[appname]/... This is also in the YAML stack definition file.
Now setting up a cron job to auto backup this tree to Google drive or Dropbox every day. All applications are therefore backed up in one easy process... As long as you have the YAML and the data, restoring a system is trivial.
Next bit is moving the media server onto it too...
Vivitur ingenio, caetera mortis erunt
- jack
- Thermionic Monk Status
- Posts: 5493
- Joined: Wed Dec 29, 2010 8:58 pm
- Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
- Contact:
#64 Re: General Raspberry Pi stuff
Reading the Pi 4 boot ROM release notes at https://github.com/raspberrypi/rpi-eepr ... e-notes.md
...I noticed the following:
Network boot was added on 23/09/19 and they said that USB MSD boot would follow that, a new Beta ROM was started on 09/04/20 plus ROM updates seem to happen every few days for the Pi 4, so fingers crossed!
Edit: Have ordered my first 4GB Pi 4 from Farnell and some extras (fun stuff) for it from Banggood & AliExpress.
I've ordered the 4GB as I think that the 2GB might be a bit tight on memory for all I have planned for this machine... The 2GB Pi 4 is a LOT cheaper (unreasonably so, IMHO), so I may end up using a 2GB one for the networking & media server stuff and the 4GB one in the workshop...
Whatever ends up happening, I'm feeling that the 1GB RAM of the Pi 3B/B+ is VERY limiting for this number of apps with docker...
...I noticed the following:
...so there is hope for the near futureon the 18/11/2019, the RPi 4 boot ROM release notes wrote:First release candidate before this beta is moved to a stable release series.
- Avoid resetting TFTP prefix after retries or if start4.elf is not found.
- Add MAC_ADDRESS option which allows the OTP Ethernet MAC address to be
overriden. An VideoCore firmware update will propagate this forced
mac address to device-tree/cmdline in the near future.- Various internal refactorings to prepare for USB MSD storage boot in
the next beta-series.- Enable high-speed mode for EMMC cards.
Network boot was added on 23/09/19 and they said that USB MSD boot would follow that, a new Beta ROM was started on 09/04/20 plus ROM updates seem to happen every few days for the Pi 4, so fingers crossed!
Edit: Have ordered my first 4GB Pi 4 from Farnell and some extras (fun stuff) for it from Banggood & AliExpress.
I've ordered the 4GB as I think that the 2GB might be a bit tight on memory for all I have planned for this machine... The 2GB Pi 4 is a LOT cheaper (unreasonably so, IMHO), so I may end up using a 2GB one for the networking & media server stuff and the 4GB one in the workshop...
Whatever ends up happening, I'm feeling that the 1GB RAM of the Pi 3B/B+ is VERY limiting for this number of apps with docker...
Vivitur ingenio, caetera mortis erunt
#65 Re: General Raspberry Pi stuff
When you reach the limits of the Pi4 then Docker swarm on this is maybe the next step? https://turingpi.com/
I've been lurking on this thread and eyeing up my box of Pi3's in the corner for a lock-down project. Pi-hole looks to be a good investment for one of them, along with teaching myself python and cracking on with a bit of home automation...
I've been lurking on this thread and eyeing up my box of Pi3's in the corner for a lock-down project. Pi-hole looks to be a good investment for one of them, along with teaching myself python and cracking on with a bit of home automation...
-
- Shed dweller
- Posts: 2299
- Joined: Fri Aug 03, 2007 10:57 am
- Location: From the land of the Bodgers
#66 Re: General Raspberry Pi stuff
Interesting but having worked with hyper converged systems before, I’m not sure a 100Mbps communication bus between nodes really cuts it! Struggling to see the problem that unit is trying fix.
Only the Sith deal in absolutes.
- jack
- Thermionic Monk Status
- Posts: 5493
- Joined: Wed Dec 29, 2010 8:58 pm
- Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
- Contact:
#67 Re: General Raspberry Pi stuff
I've been playing with docker swarm too - it's desperately cool...
...however CPU power and threads are not the limiting factors in this project - I'm not mining bitcoin or doing prime factoring!
Memory is the issue - the 3 series are only 1GB max. I want simplicity, i.e. a single host if I can get away with that. Plus the Pi 4 has proper 1GB ethernet (i.e. not just hanging off the USB controller), and the USB controller is genuine 3.0, so the M.2 SATA SSD is even faster (not that that is an issue - the SSD is about reliability here, not speed).
Pi-hole, in a domestic environment, is actually pretty economic with resources, even with the 40+ devices we have - it's amazing how many bits of odd kit crop up when you have a decent managed network that's capable of seeing what's out there... every MAC address has been tracked and identified.
Kubernetes, IMHO, in a domestic environment is overkill, however docker itself has a small footprint - definitely a lightweight shim and Portfolio.io does an excellent job of managing it.
The main resource hog at the moment is the UniFi Controller. Each on their own on a Pi 3B is fine, but together the memory gets tight, and then I still need to add a media server, plus whatever gets added in the future (and there will be stuff).
So, keeping it simple and just spending the extra 20 quid or so now seems a more sensible approach.
Bizarrely, using docker really does simplify a lot of stuff. The more I learn about this ecosystem, the more I like it.
...however CPU power and threads are not the limiting factors in this project - I'm not mining bitcoin or doing prime factoring!
Memory is the issue - the 3 series are only 1GB max. I want simplicity, i.e. a single host if I can get away with that. Plus the Pi 4 has proper 1GB ethernet (i.e. not just hanging off the USB controller), and the USB controller is genuine 3.0, so the M.2 SATA SSD is even faster (not that that is an issue - the SSD is about reliability here, not speed).
Pi-hole, in a domestic environment, is actually pretty economic with resources, even with the 40+ devices we have - it's amazing how many bits of odd kit crop up when you have a decent managed network that's capable of seeing what's out there... every MAC address has been tracked and identified.
Kubernetes, IMHO, in a domestic environment is overkill, however docker itself has a small footprint - definitely a lightweight shim and Portfolio.io does an excellent job of managing it.
The main resource hog at the moment is the UniFi Controller. Each on their own on a Pi 3B is fine, but together the memory gets tight, and then I still need to add a media server, plus whatever gets added in the future (and there will be stuff).
So, keeping it simple and just spending the extra 20 quid or so now seems a more sensible approach.
Bizarrely, using docker really does simplify a lot of stuff. The more I learn about this ecosystem, the more I like it.
Vivitur ingenio, caetera mortis erunt
- jack
- Thermionic Monk Status
- Posts: 5493
- Joined: Wed Dec 29, 2010 8:58 pm
- Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
- Contact:
#68 Re: General Raspberry Pi stuff
Pi 4 has arrived from Farnell, seen here with nice heatsinks etc. from Banggood. Also POE hat from Farnell plus other bits...
Small issue with the heatsink in that the rubber sticky pads they provided for thermal coupling were actually 3M double sided foam tape - an INSULATOR! https://www.3m.co.uk/3M/en_GB/company-u ... Tape-9448A
Replaced with the correct stuff - a bit of this: https://uk.farnell.com/bergquist/gp5000 ... dp/1893450 - just as well I checked...
Onwards!
Small issue with the heatsink in that the rubber sticky pads they provided for thermal coupling were actually 3M double sided foam tape - an INSULATOR! https://www.3m.co.uk/3M/en_GB/company-u ... Tape-9448A
Replaced with the correct stuff - a bit of this: https://uk.farnell.com/bergquist/gp5000 ... dp/1893450 - just as well I checked...
Onwards!
Vivitur ingenio, caetera mortis erunt
- jack
- Thermionic Monk Status
- Posts: 5493
- Joined: Wed Dec 29, 2010 8:58 pm
- Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
- Contact:
#69 Re: General Raspberry Pi stuff
New office server up and running. Really like the Pi 4, but I'd forgotten to order the microHDMI->HDMI adapters, so had to wait for some from Amazon...
Note I'm using a standard Anker USB C charger brick (which does proper USB C), rather than the normal Pi 4 charger...
Note I'm using a standard Anker USB C charger brick (which does proper USB C), rather than the normal Pi 4 charger...
Vivitur ingenio, caetera mortis erunt
#70 Re: General Raspberry Pi stuff
wicked!
just finished configuring the pi 4b with lamp. Ported all the test databases and php stuff and it all runs as well after some spanner work under the bonnet. I wasn't at all surprised because there where serious mismatches between the advertised latest versions of mysql and phpmyadmin but it's now purring.
Response seems much sharper than the old windows machine running wamp.
I can now retire the homeplug from the windows machine and dispose of wamp.
there is probably a lot more the pi 4 can do as I don't think php, mysql and html testing is going to trouble it much. I wonder if there is any benefit taking the LMS off the pi 3 and just leaving it with squeezelite.......
just finished configuring the pi 4b with lamp. Ported all the test databases and php stuff and it all runs as well after some spanner work under the bonnet. I wasn't at all surprised because there where serious mismatches between the advertised latest versions of mysql and phpmyadmin but it's now purring.
Response seems much sharper than the old windows machine running wamp.
I can now retire the homeplug from the windows machine and dispose of wamp.
there is probably a lot more the pi 4 can do as I don't think php, mysql and html testing is going to trouble it much. I wonder if there is any benefit taking the LMS off the pi 3 and just leaving it with squeezelite.......
There's nowhere you can be that isn't where you're meant to be
- jack
- Thermionic Monk Status
- Posts: 5493
- Joined: Wed Dec 29, 2010 8:58 pm
- Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
- Contact:
#71 Re: General Raspberry Pi stuff
Indeed. How much RAM on your Pi 4?
Put docker & Portainer.io on it, then run LAMP in a container.
Then you can run LMS in another container...
...and Pi-hole in another...
...and so on...
FWIW, I'll have a docker/Portainer stack with all the setup for that in a day or so...
Vivitur ingenio, caetera mortis erunt
#72 Re: General Raspberry Pi stuff
4gb
I'm getting more and more impressed with it as an apache server.
my only gripe so far is that the red led at the front is too bright.
all the docker stuff is a bit ott for me at the moment but if I expand it's workload in the future I'll take another look.
I'm getting more and more impressed with it as an apache server.
my only gripe so far is that the red led at the front is too bright.
all the docker stuff is a bit ott for me at the moment but if I expand it's workload in the future I'll take another look.
There's nowhere you can be that isn't where you're meant to be
- jack
- Thermionic Monk Status
- Posts: 5493
- Joined: Wed Dec 29, 2010 8:58 pm
- Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
- Contact:
#73 Re: General Raspberry Pi stuff
That sounds great Ed...
One small thing to think about.
Normal uSD cards are designed for the occasional photo, not for extensive writing. If they do wear-levelling at all, it's pretty crude. Also the cell technology is not designed for high write counts.
I've had even good quality uSD cards (e.g. SanDisk) fail after a couple of months, hence my move to SSDs for real work.
However, you can get "industrial" uSD cards and cards designed for video recording - these are more expensive but use better quality cells, more standby (spare) cells and better wear levelling. It may be worth investing in a more appropriate card. Typically rated between 25x to 200x the write tolerance of standard uSD cards.
A lamp stack will write logs etc. as well as data from your apps, so I'd really be sure to have regular backups of your system...
One small thing to think about.
Normal uSD cards are designed for the occasional photo, not for extensive writing. If they do wear-levelling at all, it's pretty crude. Also the cell technology is not designed for high write counts.
I've had even good quality uSD cards (e.g. SanDisk) fail after a couple of months, hence my move to SSDs for real work.
However, you can get "industrial" uSD cards and cards designed for video recording - these are more expensive but use better quality cells, more standby (spare) cells and better wear levelling. It may be worth investing in a more appropriate card. Typically rated between 25x to 200x the write tolerance of standard uSD cards.
A lamp stack will write logs etc. as well as data from your apps, so I'd really be sure to have regular backups of your system...
Last edited by jack on Mon May 04, 2020 9:31 am, edited 1 time in total.
Vivitur ingenio, caetera mortis erunt
#74 Re: General Raspberry Pi stuff
thanks for the tips Nick.
It maybe that I've chanced upon a good sd card by accident. I've just done a bit of googling and the one I used was a Sandisk ultra 64gb which appears to have wear levelling. It also seems to be quite fast at 100MB/s. This is all new to me, I only went for the largest and cheapest card card at Ebuyer.
backup? what's a backup! snigger
It maybe that I've chanced upon a good sd card by accident. I've just done a bit of googling and the one I used was a Sandisk ultra 64gb which appears to have wear levelling. It also seems to be quite fast at 100MB/s. This is all new to me, I only went for the largest and cheapest card card at Ebuyer.
backup? what's a backup! snigger
There's nowhere you can be that isn't where you're meant to be
- jack
- Thermionic Monk Status
- Posts: 5493
- Joined: Wed Dec 29, 2010 8:58 pm
- Location: ɐılɐɹʇsnɐ oʇ ƃuıʌoɯ ƃuıɹǝpısuoɔ
- Contact:
#75 Re: General Raspberry Pi stuff
These are the same ones I use as my general uSD card for the Pis - all my audio clients have them... and they fail every few months. I have 7 Pis doing stuff in the house - one uSD card fails every couple of months...
SanDisk do indeed have wear-levelling, but it's the memory cell technology of the card which ultimately determines the number of safe write cycles.
The speed of the card is not that important for Pis - it's really about saving and reading big photos to/from the card quickly. For general Pi use, a standard C10 card is more than adequate. It's also worth noting that the Pi 4's uSD interface is twice as fast as the 3B+; likewise the uSD is no longer a shared controller and is genuine USB 3 (on the two blue ports). A USB-connect SSD is at least an order of magnitude faster than a uSD card.
Should also point out that there's no standard for wear-levelling - each manufacturer can choose to do their own thing (or nothing at all) - so in a given price bracket/use case some are better than others.
However, for reliability, go for the industrial or video/security camera specified ones.
Vivitur ingenio, caetera mortis erunt