Delete the oldest files from a folder as long as it exceeds a certain size

Samples

Note that you have to launch using “source“

  • Delete the oldest files in the current folder (./) as long as it takes up more than 96MB:
    source ./clean_custom.sh --path ./ -l 9600000
  • Delete older files from temporary folder (/tmp/) as long as it takes more than 2GB:
    source ./clean_custom.sh --path /tmp/ -l 2000000000

Script source code

#!/usr/bin/env bash                                                              
PATH_TO_CLEAN=                                                                   
NUMBER_FILES_TO_DELETE_EACH_LOOP=1                                               
SIZE_LIMIT=2000000000                                                            
                                                                                 
# ----------------------------------------------------------------------------   
# usage:                                                                         
usage()                                                                          
{                                                                                
    echo "Clean directory: while size of a dir > limit, oldest files first."
    echo "Usage: ${filename} [-p|--path path] [-s|--max-size size] | [-h]"
    echo "    -p|--path: path to clean"            
    echo "    -l|--limit: max size for the folder (must be > 0)"
    echo "    -h|--help this help"                 
}                                                                                
                                                                                 
# ----------------------------------------------------------------------------   
# handling arguments:                                                            
args=("$@")                                                            
filename=$(basename -- "$0" | sed 's/\(.*\)\..*/\1/')        
while [ "$1" != "" ]; do                                     
    case $1 in                                               
        -p | --path ) shift              
                      # stop if path doesn't exist:
                      if [ ! -d "$1" ]; then
                          echo "Path not found: '$1'"
                          usage
                          return 1
                      fi
                      PATH_TO_CLEAN=$1
                      ;;
        -l | --limit ) shift             
                       SIZE_LIMIT=$(echo $1 | bc)
                       if [ $SIZE_LIMIT -le 0 ]
                       then
                           usage
                           return 1
                       fi
                       ;;
        -h | --help ) usage              
                      return
                      ;;
        * ) usage                        
            return 1 
    esac                                                     
    shift                                                    
done                                                                             
[ -z "$PATH_TO_CLEAN" ] && echo "Path empty" && usage && return 1
echo "Cleanin dir: '$PATH_TO_CLEAN', size limit=$SIZE_LIMIT" 
# ----------------------------------------------------------------------------   
# handling arguments:                                                            
while [ 1 ]                                                                      
do                                                                               
    s=$(du -sb $PATH_TO_CLEAN | cut -f1 | bc)                
    if [ $s -gt $SIZE_LIMIT ]                                
    then                                                     
        find $PATH_TO_CLEAN -type f -printf '%T+ %p\n' | \
            sort -nr | \
            tail -$NUMBER_FILES_TO_DELETE_EACH_LOOP | \
            cut -d' ' -f 2- | \
            xargs -I {} rm -f {}
    else                                                     
        break                            
    fi                                                                                                                                                                                                                                                      
done                                                                             
return 0

Django scripting: “AppRegistryNotReady: Apps aren’t loaded yet” solution!

If you want to make a simple script that wants to import your application built on top of the Django framework, you would probably do this code:

import django
from app.models import MyModel

You will probably get this error:

django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet

.

Don’t panic!
The solution is to run your application’s setup() before imports, as follows:

import django

if __name__ == '__main__':
    django.setup()
    # import AFTER setup
    from app.models import MyModel
    # from now I can access MyModel!!

Python: compile and run multiple versions without collisions

You have to go find the source code for the Python version that you want.

Example, run an “old” Python 3.6, go here and take the one that we want.

Then get the source code and compile it:

mkdir ~/source ; cd ~/source
wget https://www.python.org/ftp/python/3.6.13/Python-3.6.13.tar.xz
tar xvf Python-3.6.13.tar.xz
cd ~/source/Python-3.6.13
./configure && make
sudo make altinstall

“Et voilà”!

~/source/Python-3.6.13$ python3.6
Python 3.6.13 (default, May 21 2021, 17:12:12) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Asynchronous Python TCP server. And its C# Unity client!

Two very short examples to get you started, which send and receive “pure” binary = very little bandwidth, with a persistent connection.

Here are two send-receive samples which should allow you to do all your binary sendings:

  1. C#: the client sends a byte, which corresponds to a boolean, to say if it is in big or little endian;
  2. C#: the client sends a message encoded in UTF-8 (yes I found the solution that works!);
  3. Python: the server reads this boolean;
  4. Python: the server reads the message and says it (under Windows only, remove the code if you’re under Linux);
  5. Python: the server sends an unsigned integer, then two floats;
  6. C#: the client reads the unsigned integer then two floats.

With that, you have enough to understand and make all the streams you want!

Asynchronous TCP Python server

import asyncio
import struct
from asyncio import StreamWriter, StreamReader
import pythoncom
import win32com.client as win32_client
HOST = '192.168.1.31'
PORT = 9696
async def handle(reader: StreamReader, writer: StreamWriter):
    is_little_endian = False
    buffer = bytearray(100)
    addr = writer.get_extra_info('peername')
    print(f"Connected with {addr!r}")
    is_little_endian, = struct.unpack_from(
        '?', await reader.read(struct.calcsize('c'))
    )
    print(f'{is_little_endian=}')
    data = await reader.read(4096)
    message = data.decode('utf8')
    pythoncom.CoInitialize()
    speak = win32_client.Dispatch('SAPI.SpVoice')
    speak.Speak(message)
    print(f"Received {message!r} from {addr!r}")
    print(f"Send: {message!r}")
    float1 = 1.1
    float2 = 2.2
    struct.pack_into(
        # =: native order, std. size & alignment
        # H: unsigned short
        # f: float
        "=Hff",
        buffer, 0, 1, float1, float2)
    writer.write(buffer)
    await writer.drain()
    print("Close the connection")
    writer.close()
async def main():
    server = await asyncio.start_server(handle, HOST, PORT)
    print(f'Serving on {server.sockets[0].getsockname()}')
    async with server:
        await server.serve_forever()
asyncio.run(main())

C# Unity Client

using System;
using System.IO;
using System.Net.Sockets;
using UnityEngine;
public class Connexion : MonoBehaviour
{
    public string server;
    public string message;
    public ushort port;
    private void Start()
    {
        // working sample to send text:
        byte[] data = System.Text.Encoding.UTF8.GetBytes(message);
        byte isLittleEndian = BitConverter.IsLittleEndian ? (byte)1 : (byte)0;
        TcpClient client = new TcpClient(server, port);
        NetworkStream stream = client.GetStream();
        // Send the message to the connected TcpServer.
        stream.WriteByte(isLittleEndian);
        stream.Write(data, 0, data.Length);
        Debug.Log($"Sent: {message}");
        // read sample
        BinaryReader reader = new BinaryReader(stream);
        uint len = reader.ReadUInt16();
        var x = reader.ReadSingle();
        var y = reader.ReadSingle();
        Debug.Log("len=" + len);
        Debug.Log($"x={x}, y={y}");
    }
}

For the record, these two examples seem simple, but they took me a long time, and I had no answer after 3 weeks on stackoverflow…

Django: how to check if we are in debug mode in the template

You will find the information easily, and it seems very simple: in your template, just do:

{% if not debug%}I'm in debug mode!{% endif%}

This won’t work, this is not enough!

In your settings.py file, you must configure the IP’s correctly, which specify that you are / or not / in “development” mode:

INTERNAL_IPS = ['127.0.0.1',]

(Note that this code can be optimized in a way that depends on the environment, for example I made a settings.py which takes this into account).

Django: how to make customized error pages (404, 500, etc.)

When you want to make custom error pages, it’s very simple … once you know how to do it!
The documentation is not very clear on this point…

Here is a summary of my experience, to make custom error pages (404, 500, etc.).
You should first know that errors are called functions (= you cannot do this via generic views).
Then the documentation gives an example, but it’s not enough.
Thanks to Django code shortcuts, you have the function render() to which you can pass a template and a context (= therefore variables).
I used this to create a dictionary that contains the errors, and pass them in a context (= dictionary) with the title and content keys.

Here is the view code that displays “cleanly” the error:

from django.shortcuts import render
from django.utils.translation import gettext_lazy as _
VIEW_ERRORS = {
    404: {'title': _("404 - Page not found"),
          'content': _("A 404 Not found error indicates..."), },
    500: {'title': _("Internal error"),
          'content': _("A 500 Internal error means..."), },
    403: {'title': _("Permission denied"),
          'content': _("A 403 Forbidden error means ..."), },
    400: {'title': _("Bad request"),
          'content': _("A 400 Bad request error means ..."), }, }
def error_view_handler(request, exception, status):
    return render(request, template_name='errors.html', status=status,
                  context={'error': exception, 'status': status,
                           'title': VIEW_ERRORS[status]['title'],
                           'content': VIEW_ERRORS[status]['content']})
def error_404_view_handler(request, exception=None):
    return error_view_handler(request, exception, 404)
def error_500_view_handler(request, exception=None):
    return error_view_handler(request, exception, 500)
def error_403_view_handler(request, exception=None):
    return error_view_handler(request, exception, 403)
def error_400_view_handler(request, exception=None):
    return error_view_handler(request, exception, 400)

Once the views are done, you have to go to the main declaration of your views. This is the urls.py file at the root of your project. If you put the code elsewhere, it will be ignored.

In this file, declare your functions that handle the errors:

handler404 = 'app.views.errors.error_404_view_handler'
handler500 = 'app.views.errors.error_500_view_handler'
handler403 = 'app.views.errors.error_403_view_handler'
handler400 = 'app.views.errors.error_400_view_handler'

And finally, in your templates, create a file errors.html in which you will code your HTML page which shows the error in a clean way.
Note that you have in the context the variables {{ title }} and {{ content }} which are respectively the title and the detail of the error.

Now, how, in “development” mode, test these pages? In practice you cannot, because in development mode, an error displays debugging information, and not your pages!

The solution: make a URL that “simulates” the error. Example with 404: add in your main url’s file: path('404/', error_404_view_handler), and then display the appropriate URL in your browser, ie http://localhost:8000/404/ and you will see the error!

I’m sorry my english is always perfectible, if I’ve made some mistakes, please let a comment and I’ll correct this post. Thanks a lot!

Django >= 2.1: how to make a customized administration, the “clean” way

How to make easily an administration interface for the whole project

On versions of Django < 2.1, it was impossible to overload the administration “properly”.

  • Create a file admin.py at the root of your project (“at the root”, because it is “global” for the whole project)
  • Put that code:
    from django.contrib import admin
    class MyProjectAdminSite(admin.AdminSite):
        title_header = 'My project Admin'
        site_header = 'My project administration'

    Of course, change “MyProject” to the name of your project…
  • In settings.py, replace 'django.contrib.admin' by 'my_app.apps.MyAppAdminConfig',
  • In the file apps.py of your project, hence my_project/my_app/apps.py, declare the administration like this:
    from django.apps import AppConfig
    from django.contrib.admin.apps import AdminConfig
    class MyAppConfig(AppConfig):
        name = 'my_app'
    class MyAppAdminConfig(AdminConfig):
        default_site = 'admin.MyProjectAdminSite'

And… that’s it!