Tiny Tiny RSS: Community

Overhaul: TTRSS Needs a Data Directory


#1

Hey all,

I was looking to help the Linuxserver group update their Docker container for TTRSS, however I’ve run into an issue: (I can’t post links, but it’s issue 16 on their repo).

It seems that the upgrade path for TTRSS is a git pull, and since dynamic files are scattered all through the directory a pretty in-depth gitignore is required.

This makes updating in a container very difficult, since at build time the TTRSS directory for the user doesn’t exist.

My suggestion is to revamp data storage, and create a data folder that all dynamic content is kept in. Cache, plugins, deployment, config, etc. Basically, have that data folder be completely empty at install time, and only populate once the installer starts running. This is how most new web software works, like Nextlcoud, etc.

I understand this could be a big undertaking, and require a scripted update method for users, but this could be highly beneficial going forward.

Thanks.


#2

i’m not going to change how things work for everyone because of a docker container i don’t provide.

also, i find this whole idea very questionable tbh.

most “new web software” is irredeemable disfunctional garbage. just because something is new doesn’t mean it’s any better. find a better argument.


#3

It isn’t that scattered. I’ve got it running on read-only containers under docker swarm with no issues using:

VOLUME /code /code/lock /code/feed-icons /code/cache /code/cache/js /code/cache/starred-images

(/code is the git checkout.) You could probably even shrink that further with symlinks or creating cache/* on container start.

Honestly I was surprised how easy it was to containerize, even with splitting it into tt-rss and tt-rss-update containers. Guess I should post my code somewhere.


#4

Just out of curiosity: Why do you run ttrss with docker and swarm if you could just run it the “standard way” and do some git pull on a regular basis? What are the advantages?


#5

containers are easier to manage if you don’t want to shit up your system with third party apps and their dependencies

there’s disadvantages to this approach too of course


#6

So this is helpful, thank you. I took a look and it seems many of the items in the .gitignore don’t exist for me, (especially the hidden files) but I’m not sure if that’s due to changes in the software (web.config, maybe?) or just because I don’t have certain plugins installed.

Will tt-rss create what it needs on start int he /cache folder of that’s mounted to an external volume? For instance, /cache/images or /cache/feeds? And I assume you keep local plugins inside your image since you don’t have those volumes.

My biggest concern is the “things that don’t exist.” My plan had been to create a /data folder at build time, then move the contents of config.php, cache, lock, plugins.local, and themes.local into /placeholder. Then a user can mount /data, and a script at boot would see if /data was empty, and if so, copy /placeholder into it before starting.

The problem is I don’t want to have any users pulling the repo and one of those other files from gitignore getting created and the system losing it’s state.

I’m thinking my plan will work given that you’ve been running a while, but knowing if these files are needed would be helpful (along with if /lib/htmlpurifier needs to be outside the container):

  • /deploy.exclude
  • /deploy.sh
  • /messages.mo
  • .idea/*
  • tags
  • lib/htmlpurifier/library/HTMLPurifier/DefinitionCache/Serializer/*/*ser
  • /.save.cson
  • web.config
  • /.tags*
  • /.gutentags

Thanks!


#7

i’m not sure why are you acting so surprised that things mentioned in .gitignore don’t exist in the repository
it’s what this file is literally for

also, i’m not surprised that someone who can’t even recognize ctags is coming here asking for grand overhauls for a 10+ year old project. how old are you, op? were you alive before web 2.0?


#8

Every time you look at a system update containing 400 php libraries, you know what the advantage is. No cruft laying around, no random conflicts between apps, etc. (And images make a much easier rollback)

The other big advantage is a lower attack profile. If you break php (and its happened to me in the distant past courtesy WP…) on a traditional system, you have the system.

If you get into my container, you have a mostly read-only minimal system with noexec/nodev/etc on all of the read-write volumes. So you could push up a script or something, but that is about it. (Bonus, your attack has to pass traefik and nginx sanity checks before you have any hope of touching php.) The container network is isolated to the proxies and db (also r/o) so it can’t touch the host or any unrelated services. And doing anything too exciting will get it killed by cpu/memory limits.

That reminds me, I need to doublecheck that php won’t run scripts from those volumes.


#9

yeah because it’s inside the container :thinking:


#10

That is a good thing. “I wonder if this library is needed by or if it was from that ancient installation of pocket? Better keep it just in case.” (Oh, goody, its from multiverse and hasn’t been updated through the past 84000 critical CVEs…)

When I toss out a service, all of it’s cruft goes with it.


#11

that’s really neither here nor there

for example imagine if you will a docker image. it’s a stock three year old ubuntu, unpatched, made by some drooling imbecile

now imagine a bunch of those

e: i think we’re hijacking this thread :face_with_raised_eyebrow:

yeah exactly this but in multiple containers


#12

Ahhh there’s that personality I heard about on Reddit. Anyway, I never said they don’t exist in the repository. They don’t exist on my install. Just because they don’t exist in mine now doesn’t mean I or someone in the future will install some add-on or turn on some option that I’m not aware of that works outside of the plugins or themes folders that adds one of those files and breaks the container on restart. I don’t want to put out an irredeemable or dysfunctional container, so I want to know what I need to include in the volume mapping before I create it.

As for the rest, I simply cut/paste the items that did not exist on my system, and my grand “overhaul” consists of moving a few data directories and a config file. And if it matters, my guess would be we’re close to the same age.


#13

I always wonder why container creators on the HUB don’t auto-rebuilt based on the FROM or monthly. Annoys the crap out of me. Lazy.


#14

We are hijacking but thats cuz the grownups are talking. :bearded_person:

You’re assuming the container has a long lifecycle and does dumb things. My oldest container is 2 weeks old (postgres:alpine) and every deploy involves pulling down new images and updating if needed. For abandoned OSS images, it’s easy enough to pull down the build env and rebuild it from a newer base (or even add a system update to the build.)

My direct internet-facing images generally don’t have anything except the executable (the only thing I like about go) - eg https://github.com/containous/traefik/blob/master/Dockerfile

Even if you can crack that, you have to do everything through that executable (r/o ftw) because the container consists of that executable and nothing else. (And it’s interface to the outside system is all r/o as well)


#15

yeah if you make and maintain your own good containers it’s all good, i’m talking about third party stuff which i’m always assuming is horrible broken garbage


#16

Anything tagged official is required to (among other things):

Ensure that security updates are applied in a timely manner.

(https://docs.docker.com/docker-hub/official_images/)

So it isn’t too terrible. Not as bad as gems and ppa and multiverse and… etc.


#17

Would you be able to share your docker file that you use to build your image?


#18

It needs to be fronted by something (I use nginx-alpine) but https://gist.github.com/disconn3ct/bd67ece676f1497975a96934f59434df


#19

i think your problem partially stems from trying to use trunk .gitignore as something it is not. it’s the list of files i don’t want to see in git status, nothing more, nothing less. this list including stuff that may appear at runtime on someone’s install is incidental and doesn’t really concern me in any way.

that said, the list of directories where stuff does appear at runtime is fairly obvious. if not, it has literally been already posted upthread. there’s also not going to be any overhauls that tt-rss i’m sure doesn’t actually need, so in all honesty i’m not sure what we’re discussing here at this point, other than bad docker containers

also, maybe you should try making your stuff less fragile so that tags appearing at application root (somehow) wouldn’t break container booting (somehow)

i’m sorry creepy hugbox people dislike me for not mollycoddling people on my forum


#20

Thanks! I am creating a Dockerfile now that uses a single mapped /data directory so that the latest master can be pulled at build time (instead of having to do a git pull). This way an old version of the container would always have the same version of TT-RSS instead of the pull happening post-build. I’ll post it up when I’m done, likely later today.

edit
BTW, why don’t you do the recommended configure on gd?