Contribute to wappalyzer

Posted on 20. Mai 2020 Comments

Wappalyzer is an open source website analyzer written in node.js. It basically just parses a big json file and uses regular expressions to find patterns in websites HTML, CSS, JavaScript and Server Headers.

  1. Fork the repository https://github.com/AliasIO/wappalyzer
  2. Clone the fork to your computer
  3. Install Docker
  4. ./run links
  5. Write Regex
  6. Check with regex101.com or similar tool
  7. Add valid json to apps.json
  8. Add a 32×32, 64×64 PNG or SVG to the icons folder
  9. Commit to another branch on your fork
  10. Push
  11. Create Pull Request, showing that it’s a relevant project: 1k+ stars on GitHub, Pages using it etc

I just created a pull request for Alpine.js, A rugged, minimal framework for composing JavaScript behavior in your markup, that I like to pull in when Vue or React would be overkill.

Deploy with envoyer and artisan

Posted on 20. Mai 2020 Comments

I just revived JustParks Envoyer Deploy package and updated it for Laravel 5.5+ (handle() and fire() – both work now), 6 and 7. I haven’t written it, so all the credit goes to Dayle Rees/JustPark.

Updated envoyer:deploy package on packagist

As you can tell from the README, Install like so:

composer require repat/envoyer-deploy --dev

Then publish the config file by executing this and selecting the number that says JustPark\Deploy\ServiceProviders\EnvoyerServiceProvider.

php artisan vendor:publish

In the envoyer.php config file, fill in the unique ID that comes after the /deploy in the link you can find in the Deployment Hooks tab in envoyer, e.g. https://envoyer.io/deploy/4aLDdfsfsd4s6fSzeKGNfakekey75R45wOwTQULEDJNrj

You can now deploy with

php artisan envoyer:deploy

Sourcecode for repat/envoyer-deploy on GitHub

Digital Nomad Guide to Self-Isolation in New Zealand during the Corona Virus Outbreak 2020

Posted on 18. März 2020 Comments

Update: Well, as expected this post didn’t age well. New Zealand closed its borders to non-residents.

As anything written about the Corona Virus outbreak doesn’t age well, consider this article was written on Thursday 19th March 2020 and I entered NZ on Tuesday 17th March 2020. Things might’ve drastically changed by the time you read this, so do some more research! Also, this is not legal advice, obviously.

When you’re looking for (temporary) refuge from the recent Corona Virus outbreak and you’re in South-East Asia, New Zealand looks like a good option: it’s a civilized democracy with great healthcare, friendly people, a fairly reasonable government, good internet connection and low amount of confirmed cases so far. Even if you’re „stuck“ here for longer, I’m sure you won’t run out of things to do. Alternatives to New Zealand could be Australia and Singapore.

However, you will need to self-isolate for 14 days, so first things first, here’s the official NZ government website about how to self-isolate. There are always idiots that think they don’t have to follow these rules, but NZ police will do spot checks, as I can tell you also from personal experience. And while the rules are not as strict as in e.g. Singapore, they might also force quarantine or deport you if you break the rules or even change them while you’re still isolating.

Things to consider:

  1. A lot of nationalities (most of Europe, US, Canada) need to sign up for NZETA, the Kiwi version of US ESTA, a visa waiver program. It can take up to 72h and costs only $47 NZD so do it now, just to have the option. It’s valid for 2 years, so it’s not like it’s wasted money. For me it took literally one minute to get the confirmation. You have the right to stay 3 months with this or more if e.g. you’re a British Citizen.
  2. You need a return flight to either your home country or a place you have either a visa for or don’t need a visa. Check with passportindex.org for your passport. Common destinations are Australia, as flights are usually < $100 but you might need a visa, or visa waiver ETA for that too (not for transit) – some countries just need to sign up for free. However, this can also take 2 days or not arrive at all. Another option that’s visa free for a lot of passports is Singapore. You need this at check-in to board the flight to NZ.
  3. A lot of flights are already cancelled and more will probably follow as Air New Zealand for example announced a up to 85% cut of their routes. It seems like other Airlines will do the same as they really struggle at the moment. Chances are, the earlier you decide to come the easier it will be.
  4. New Zealand is far away from pretty much everything and it might be difficult to go back home, especially to Europe where no direct flights are available and you rely on transit hubs such as DXB, BKK or SIN. If something happens to your loved ones far away, it might take days and multiple hops and cancelled flights (so, lots of cash) to even get there in the future. Likewise, if something happens to you (and be it something unrelated to Covid-19, like a car accident), chances are your loved ones can’t come see you, certainly not for 14 days. There are however direct flights to the US possible.
  5. You will be asked if you’re sick before immigration, fill out an extra Covid-19 Form and they have a thermometer although they didn’t take peoples temperature when I arrived. That might have changed.
  6. There’s Uber and Uber Eats. When you arrive in Auckland AKL airport, the Uber pickup station is on your left. Take the exit next to McDonalds. It’s probably a more sensible option than public transport. You can let the Uber Eats delivery person know in the app to leave the food in front of the door or the lobby because you’re self-isolating and track when it arrives in the app.
  7. You can stay in Hotels, AirBnbs (try to get self-checkin) and (for now) camper vans, if they include a shower and toilet (so you don’t share public facilities). However, it seems you have to stay in one place and not take a road trip like these idiots, potentially infecting other people on camp grounds. You also have to inform the host that you’re self-isolating and not everybody will let you. The same goes for camp grounds/trailer parks.
  8. You are allowed to go outside to go e.g. grocery shopping but limit interaction both in distance (2m+) and time (< 15min). Obviously do this the least amount possible and buy lots in one go.
  9. Even though you could grocery shop, there’s online grocery shopping at Countdown.co.nz and New World with either pickup or delivery. They even have a Corona mode now, where they leave the food in front of the door. Obviously delivery is to be preferred but they are receiving a lot of orders nowadays and it can take 3-7 days until you get a delivery window. And just picking up a pre packed order is better than walking through the aisles. You might need to have a NZ telephone number to sign up and you can get SIM cards at the airports.
  10. Consider e.g. nightly walks, when less people are outside anyway. It’s important to take this seriously to make sure you and this wonderful host country stay safe, so stay at home as much as possible but also consider your mental health, getting some fresh air in and some sun light (while it’s still late summer), and read the CDC mental health guidelines as well as the ones from the NZ Minitry of Health.
  11. If you show any symptoms or feel unwell, call 0800 779 977 and also let your hotel, AirBnb host or camper van company know so they can be extra careful cleaning before the next person moves in.

Here’s the leaflet you will get at the airport:

Last but not least, chances are you are in Bali: There’s an Emirates flight directly from DPS-AKL at ~4 in the afternoon (coming from DXB), which has 20MB free WiFi and it’s $16 USD for the whole flight. If the login page doesn’t open go to http://172.19.248.2

Please honor the Kiwi hospitality and stay safe!

Laravel QueueWorker: database not configured

Posted on 4. November 2019 Comments

It doesn’t matter which queue system you’re using: Redis, a relational database such as MySQL or a cloud solution like AWS SQS ; you can always store failed jobs in a normal database table. This configuration is done in config/queue.php in the last array:

'failed' => [
    'driver' => 'database',
    'database' => 'default', // NOTE: connection, not database!
    'table' => 'jobs_failed',
],

The key is to put in the connection name, not the database name for ‚database‘. If you don’t you will get the error message „Database default not configured“ every time a job fails instead.

Check out my Laravel Job Eloquent Models.


Laravel Dusk: The currently authenticated user is not who was expected.

Posted on 31. Oktober 2019 Comments

If you’re having trouble using $browser->loginAs($user) it might be because the SESSION_DRIVER is set to array. So even this simple code will fail:

$this->browse(function (Browser $browser) {
    $browser->loginAs(User::first())
            ->assertAuthenticatedAs(User::first());
});

Error: The currently authenticated user is not who was expected.
Failed asserting that two arrays are identical.

/vendor/laravel/dusk/src/Concerns/InteractsWithAuthentication.php:102
/tests/Browser/AuthTest.php:162
/vendor/laravel/dusk/src/Concerns/ProvidesBrowser.php:67
/tests/Browser/AuthTest.php:163

Write a .env.dusk.local (or where ever you’re starting the tests from) and use a persistent driver like SESSION_DRIVER=database or SESSION_DRIVER=file

Crawling with Web Storage in PHP

Posted on 28. September 2019 Comments

Some pages have an extra screen before the real site appears, e.g. for age verification (like porn, for now anyways, or drug sites) or country/currency selection (like travel and big company sites). As far as I researched this, sometimes the selection is stored in a cookie, sometimes in Web Storage.

Crawling with Cookies is no problem with Guzzle.

But for example, at https://cannabis.wiki the age verification is stored in the Local Storage as you can see in the Chrome DevTools:

image

Using spatie/crawler we can inject a Browsershot (puppeteer) instance, that sets the localStorage key ageConfirmed through custom JavaScript:

$js = "localStorage.setItem('ageConfirmed', '1');";
$browsershot->setOption('addScriptTag', json_encode(['content' => $js]));

However, the code is only injected after the side is already loaded. Therefore we have to use JavaScript to reload the page:

$js = "localStorage.setItem('ageConfirmed', '1');location.reload()";
$browsershot->setOption('addScriptTag', json_encode(['content' => $js]));

You can test this on the console e.g. by using ->save($pathToFile); and have a look at the screenshot to see if it worked.

Connect SequelPro with DigitalOcean Managed MySQL 8.0.X

Posted on 19. September 2019 Comments

ALTER USER 'doadmin' IDENTIFIED WITH mysql_native_password BY 'secret_password';

Change ’secret_password‘ to the password given to you by DigitalOcean.

The reason behind this is that SequelPro and other MySQL Clients only work with the old (5.X) way of MySQL authentication vs. the new (8.X) caching_sha2_password. It has nothing to do with the MySQL version itself as Sequel Pro seems to work just as well with version 8.X. You can find more information in the MySQL documentation.