Complicated (maybe) way to upload images to S3 with my Laravel app

Well, I have this travel blog called Auringon alla (under the sun in english). Last year I wrote the whole thing with Laravel and we liked it a lot.

Now for this year I needed to make some upgrades and make the image upload process better for user (me and my wife). So here is the complicated (maybe) way that I upload photos and at the end they are stored in S3.

The blog

First lets introduce the blog Auringon alla (under the sun). We used WordPress looong time and it was getting a chore and when on holiday we do not like chores so we decided it is time to make this blog better experience for us and I hope for readers.

We have two different main sections: Blog and Trips (also single content pages etc. but lets focus on those two)

Blog

Blog contains basic blog posts like WordPress did. It is quite simple piece of code so nothing odd there.

WordPress blog posts were imported also since they were mainly html that was quite easy job to do.

Trips

This is the "bread and butter" of this site. Every trip has its own page and trips contains day. Trip page contains list of days and blog posts that are related to the trip. Day contains content which is the most important part of this site.

In WordPress every day was one huge blog post. Sometime it took 1-2 hours to write it - on holiday it is quite a long time to write blog post. Now every day is like one small microblog. Day contains updates and it evolves throughout the day and gets filled in on the next day. Blogging is like posting pics/stories to Instagram/Mastodon.

The new way of adding content to the day is much better and it does not feel like a huge job anymore. We can add content after it has happened or when we are drinking wine/coffee. So much better than the old way.

The problem

Image uploading worked fine on the first big trip (2 week in Greece).

Problem was that I used native file uploader so no cool progress bars and sometimes we did now know is the image uploading or not. Mobile internet is not so good at the times in Greece. So there is that fun thing also.

The solution

Is... complicated (maybe) way to upload images. Well it is complicated behind the scenes but for user it is quite simple.

I tried few different methods for this and even coded my own implementation of progress bars but then I came to my senses and abandoned that since it was not working great on mobile.

Then I started testing different solutions and ended up using Uppy.

Userland

So we mainly use our travelblog via iPhone homescreen shortcut. It is like an app but not really. It works great! Here are the steps that user does when uploading new content.

First we need to choose content type (text, images or link). Then we can select images using Uppy and send them to server.

After that we show the uploaded images to user and ask them to add description. This is nice to add since it is images alt text and title.

After that we can add text and choose when this happened.

And thats it. New content is added to the day.

Behind the scenes

This is probably the part that you want to read. How the app does things. I think we need to break things apart.

1. Uppy image uploading

This is the first step. Handle Uppys image uploads. I have endpoint that handles Uppys POST request. I decided to make TemporaryContent model for this since I do not know Content models ID yet. This is probably unnecessary since I could just upload image to server and return url. Now I'm using Laravel-Medialibrary to handle saving the file.

1$tempContent = TemporaryContent::create();
2$tempContent->addMediaFromRequest('file')->toMediaCollection('images', 'uppy');
3 
4return [
5    'tempcontent' => $tempContent->id,
6    'url' => $tempContent->getFirstMediaUrl('images'),
7];

2. Attaching image to Content and inserting alt texts

So Uppy has moved images to server and user has inserted descriptions for images. We receive IDs for TemporaryContent and texts.

I loop through IDs and here is the stuff that I do with single TemporaryContent ID.

 1// Get the TemporaryContent model
 2$tempcontent = TemporaryContent::where('id', $tempcontentid)->first();
 3 
 4// Get image description for that ID
 5$imagetext = $request->input('imagetext_' . $tempcontentid);
 6 
 7// Get the image from medialibrary
 8$tempcontentImage = $tempcontent->getFirstMedia('images');
 9 
10// At this point we know the final Content model since it is created, lets move the TemporaryContent image to real Content
11$spatieImage = $tempcontentImage->move($content, $content->type, 'media');
12 
13// If there is text for the image set custom property
14if ($imagetext != null) {
15    $spatieImage->setCustomProperty('alt', $imagetext);
16    $spatieImage->save();
17}
18 
19// Delete TemporaryContent
20$tempcontent->delete();
21 
22// Create job that moves images from local disk to S3 (5min delay so conversions are surely done)
23CreateMovingImageJobsJob::dispatch($spatieImage)->delay(now()->addMinutes(5));

3. Move images from local disk to S3

So we dispatched a job on step 2. At this point we have our image and conversions on local disk. Nobody but me (and readers) notices that the content has images served from local disk.

Since we use images straight from our phones they might be quite large. I wanted to start using S3 to store them so I save some space on my VPS.

CreateMovingImageJobsJob dispatches even more jobs. Poor Laravel Horizon has to do a lot of work.

 1// Disk names
 2$diskFromName = 'media';
 3$diskToName = 's3-media';
 4 
 5$media = $this->media;
 6 
 7// Lets figure out the folder where the media is
 8$mediaPath = PathGeneratorFactory::create($media)->getPath($media);
 9$diskFrom = Storage::disk($diskFromName);
10 
11// All files from the directory (including conversions)
12$filesInDirectory = $diskFrom->allFiles($mediaPath);
13 
14// Every image needs to be moved and we dispatch new MoveContentImageJob for every image
15$jobs = [];
16foreach ($filesInDirectory as $file) {
17    $jobs[] = new MoveContentImageJob($diskFromName, $diskToName, $file);
18}
19// We need to change disks on db
20$jobs[] = new SaveContentImageLocationDataToDB($media, $diskToName);
21// Delete old folder from disk
22$jobs[] = new DeleteMovedContentImageFolderJob($diskFromName, $mediaPath);
23 
24// Chain and dispatch job
25Bus::chain($jobs)->dispatch();

3.1 .MoveContentImageJob

This job simply moves image from local disk to s3

1$diskFrom = Storage::disk($this->diskFromName);
2$diskTo = Storage::disk($this->diskToName);
3 
4$diskTo->put(
5    $this->file,
6    $diskFrom->readStream($this->file)
7);

3.2. SaveContentImageLocationDataToDB

This just edits disk and conversion_disk to S3 disk.

1$this->media->disk = $this->diskToName;
2$this->media->conversions_disk = $this->diskToName;
3$this->media->save();

3.3. DeleteMovedContentImageFolderJob

This just deletes the folder from local disk.

1$diskFrom = Storage::disk($this->diskFromName);
2$diskFrom->deleteDirectory($this->path);

So does this work?

It seems to be working just fine. At least my testing has been succesfull and everything seems to be working. Some error and exception handling is probably needed and it needs some finessing but hey at least it seems to be working.

There might be some cases that image is not found but I think that is not a big deal and it will be a very short thing to happen.

I might do some refactoring 🤔