![]()
Broken links are bad for business or reputation. Most visitors consider stale hyperlinks to be a sign of laziness, carelessness and disrespect them. Unfortunately, if you have a large website with thousands of external links or let say a VLE with hundreds of courses and lots of external references, it's really hard to identify what external links became dead, and it's even harder to fix them because you need to know the exact location of the broken links.
I am going to give you a possible solution specially designed for our Moodle site, but you can use it on other websites as well. It is not perfect and it needs some human supervision (not 100% automated), but it is free and pretty much customisable.
If you are looking for a tool to do the job, you would find quite a lot, but when I tried to use them (even paid versions) they were not good enough for our purpose.
Let me explain. The problem we have to deal with is:
There is an existing plugin for Moodle sites, but we cannot install plugins if they are not on our provider's matrix, so, unfortunately, it is not a valid option for us. We also tried one of the industry-leading SEO tools (and now you understand the title hopefully) and it was very promising, but it was a little bit too much. The tool was very accurate, I would say too accurate as it 'clicked' on every single possible item on the page to scan the whole VLE. Therefore it generated thousands of log entries and we had to be very careful what kind of permission we gave to the 'robot' user. Also, it was complicated to use it as it belongs to the marketing department and we wanted something we can use on our own. I have also tried different freeware products a lot, but most of them worked fine with static websites only, or not password protected sites only. No good for us.
And on a random Sunday afternoon, I realised that I have got all the tools and skills I need to create my own solution.
This is what I came up with:
Step 1: The user account
You most probably need a brand new role. You can start from the student archetype and remove 'dangerous' capabilities, or you can start from nothing/blank and add the necessary capabilities. I prefer the second method as you add only the really necessary ones.
At least you need the following settings:
Step 2 : generate URL's to check
This is a MySQL SELECT statement to give you all the pages, books (with their chapters) and lessons (with their subpages) within a course or a course category.
MySQL
And the result is something like this:
https://YOUR_OWN_MOODLE_URL/mod/book/view.php?id=78238&chapterid=32344 https://YOUR_OWN_MOODLE_URL/mod/book/view.php?id=78238&chapterid=32345 https://YOUR_OWN_MOODLE_URL/mod/book/view.php?id=78238&chapterid=32346 https://YOUR_OWN_MOODLE_URL/mod/book/view.php?id=78238&chapterid=32347 https://YOUR_OWN_MOODLE_URL/mod/book/view.php?id=78238&chapterid=32348 https://YOUR_OWN_MOODLE_URL/mod/book/view.php?id=78238&chapterid=32349 https://YOUR_OWN_MOODLE_URL/mod/lesson/view.php?id=78253&pageid=15829 https://YOUR_OWN_MOODLE_URL/mod/lesson/view.php?id=78253&pageid=15830 https://YOUR_OWN_MOODLE_URL/mod/lesson/view.php?id=78253&pageid=15831 https://YOUR_OWN_MOODLE_URL/mod/lesson/view.php?id=78253&pageid=15832 https://YOUR_OWN_MOODLE_URL/mod/lesson/view.php?id=78253&pageid=15833 https://YOUR_OWN_MOODLE_URL/mod/lesson/view.php?id=78253&pageid=15834 https://YOUR_OWN_MOODLE_URL/mod/lesson/view.php?id=78253&pageid=15835 https://YOUR_OWN_MOODLE_URL/mod/lesson/view.php?id=78253&pageid=15836 https://YOUR_OWN_MOODLE_URL/mod/lesson/view.php?id=78253&pageid=15837 https://YOUR_OWN_MOODLE_URL/mod/lesson/view.php?id=78253&pageid=15838 https://YOUR_OWN_MOODLE_URL/mod/lesson/view.php?id=78253&pageid=15839 https://YOUR_OWN_MOODLE_URL/mod/lesson/view.php?id=78253&pageid=15840 https://YOUR_OWN_MOODLE_URL/mod/lesson/view.php?id=78253&pageid=15841 https://YOUR_OWN_MOODLE_URL/mod/lesson/view.php?id=78253&pageid=15842 https://YOUR_OWN_MOODLE_URL/mod/page/view.php?id=78254 https://YOUR_OWN_MOODLE_URL/mod/page/view.php?id=78257
As you can see the script gives you all the pages, books and lessons in all courses under category 51. You can add more categories or can use specific courses, it is up to you. For the first time, I would recommend a small(ish) course where you have around 100 links only, so you can practise and gain a better understanding of the process. The links are ordered by the course module id, as sometimes you have to go and see which link was checked last time without an issue and do some troubleshooting, so it is good to know the order. For example, if you see the 'frog' is struggling with id=78253, you can presume that all the smaller ID numbers have been checked without any problem.
Step 3: Automation
This tool is fun. It helps a lot when you have to do repetitive, boring tasks. You can write scripts (macros) to do the same boring steps over and over again. You can create stable robotic process automation (RPA) scripts with image and text recognition on Windows, Mac and Linux. Real fun. And most importantly it works very well.
Visit their website for more: https://ui.vision/ I use the Chrome extension. This is the script/macro I wrote and I am going to explain it in details so you can amend it to your needs.
JSON
Phew, that was interesting. I really hope you are still with me and you can create your own little Frog based on mine.
If you have several pages, books and lessons, put all the URLs into a CSV file called 'activitiesToCheck.csv' and use it with UI.Vision but click on the arrow next to the Play Macro button and choose Play Loop. You can set up how many times you want to repeat the macro, and it will use the provided rows from your CSV file one-by-one. Sorted. A working example:
I can offer one more thing: we can try everything on my dummy Moodle site.
Download the following files: ![]()
![]()
![]()
Before you click on PLAY MACRO, it is worth to have a look at the page content and analyse it a wee bit.
And this is the result you should get:
![]()
See a wee explanation below. The Title, URL and number of links are the same, so let just focus on the other fields:
Good luck and enjoy your new toy :)
0 Comments
Leave a Reply. |