Things you do once in 10 years ...

 I enjoy fixing things, setting them up, installing and configuring, breaking and testing too. And the things can be computer things and apps, and also physical things and devices and stuff. I am glad that I have digital competences so I can do most of the digital things by my own. But sometimes they also are quite challenging. 

When more than 10 years ago (almost 15 years ago, omg...) I set up this blog, one of my proudest moments was when I bought a domain and made this blog availabe thru a domain name. Instead of typing lauvadidzis.blogspot.com I got unique with my lauvadidzis.com address. And I was proud. I like computers but in life I have to million other things for living. The problem with million things to do is the constant motion of the attention. So, I got familiar with Domain zones and zone records, but not in depth. I got an understanding in what way they are important. Got an idea of A and MX records. Primarily I use domain for a email and secondly for this blog. I got in in the Google suite while it was offered for free and I got used to google services pretty much. Made different accounts for me and my "million other things" and had a nice hierarchy of what I did in my life. When I started my company I even used google aliases for company`s email and I saved pretty much money with the things I managed to do with the googles help. And then came 2021. Google basically fucked their legacy users and I had a painful time to move away from google. Well, I found a local company which gave plenty of digital platform needs for me for the same price google started to ask per account. The best part was that on that local company`s platform I was able to "re-create" those many different accounts I had previously on google. So, in the end I was paying 6 times less as if I would have to pay to google if I would stay with it. 

This was the second time when I had to work with domain name, zones and entries. But somehow I was successful with platform changes and everything worked fine. 

Yesterday I was trying to reconfigure my email program on computer and thought I can set up webmail access too for my email. Ok, got it working fine. But while I was setting it up, I started to poke domain zone settings too. And when you start poking things like that and again fall for not doing backups for the settings, things can go terribly wrong. Well, I lost few hours to figuring how to get the domain name for my website (this blog) back, but most importantly, how to get the email MX records back...

So, few notes to add here - 

when I connect this blog with a domain zones, I need two important CNAME entries - 

1. for ghs.google.com and 
2. for key:name pair.

The other thing which is important in my case is TXT records for mail delivery.

The TXT record is an SPF (Sender Policy Framework) record. SPF is a type of DNS record that identifies which mail servers are authorized to send email on behalf of a particular domain.

v=spf1: This specifies the version of SPF being used, which is SPF version 1.

include:_spf.google.com: This part indicates that the domain's SPF policy includes the SPF record of _spf.google.com. This means that the current domain's email can be sent through the servers listed in Google's SPF record.

~all: This is the last part of the SPF record and signifies the action to be taken if the SPF check fails. In this case, the tilde (~) symbol indicates a soft fail, which means that the SPF check will not result in a hard rejection of the email, but the email might be marked or flagged as potentially suspicious.

I also discovered a service mxtoolbox.com which allows to check the domain MX records. 

So, now I have a backup notes for myself in case I poke the Domain zone entries again :) I know that I will do that... Maybe sooner than after 10 more years.

Making PCR-800 working under Windows 10

 I bought PCR-800 back in 2008. So, today when I write this post, I own it for 14 years. I havent used it a lot in these years (nevertheless I substituted my main stage piano Kawai MP4 for one summer with this due to Kawai motherboard fried and I needed to replace it. Different story tho.) But when I used it, due to lack of driver support (last out of the box compatible OS is Windows 8.1 64bit) I kept it mostly in the bag and rarely connected it to my PC via Edirol FA-66 external sound card (also fried it once. Another different story tho.) using its MIDI port. Downside - I have to deal with power cable for my keyboard (but I am lazy person) instead of powering it using USB

During last Christmas break I found a time to research a little bit more about setting my PCR-800 for Windows 10. And some old Cakewalk forums had the info that Windows 8.1 drivers will work fine with one slight modification in the driver file. Without this driver my PCR-800 in the device manager was found as unknown hardware and windows did not offer any valid drivers for it.

Driver files often have their INF files which in plain text defines different details about the hardware and they are used to set the hardware up for the operating system. And that is also a problem. So, if the INF file is for Windows 8.1, then I have to modify the file so it is suitable also for Windows 10. 

First thing - download those drivers locally. 

Then - open INF file and find line which defines Manufacturer and replace the line

1. %MfgName%=Roland,NTamd64.6.2,NTamd64.7

with the line 

1. %MfgName%=Roland,NTamd64.10,NTamd64.7

Or in other words it is visible that NTamd64.6.2 which corresponds to Windows 8 is now replaced to NTamd64.10 which seemingly corresponds to windows 10. There are interesting info when I googled about these numbers, mostly all of them related to Roland hardware.

The other changes need to be done under the particular NTamdxxx section - the section name itself has to be renamed:

1. [Roland.NTamd64.6.2]

needs to be replaced with this line

1. [Roland.NTamd64.10]

I havent experimented with would it work if the driver will install if the rest of the sections (Roland, Roland.NTamd64.7 would be deleted. Cant take that for granted.

The last part is to disable the driver signature enforcement. There are some powershell and bcdedit techniques, but those did not work with my system. The last resort for me was to restart the windows 10 in advanced boot options (can be done through Update and Security dialogs or hitting F8 during the startup) and then select the option for booting the Windows 10 with driver signature enforcement turned off. That worked for me fine and I can now write my own sheet music in Sibelius.

Adding WiFi to HP Laserjet Pro M15a with Raspberry Pi B+

 I have an old Raspberry Pi B+ since 2014. Did some experiments back then, even made it to broadcast a FM radio within 50m radius, but now I`m giving to it a new, particular tasks to do - make my simple and cheap HP LaserJet to be found in the local home network. 

So, I will need a few things. Firstly, a fresh SD Card with Legacy Raspbian OS, CLI. (While installing with Raspberry Pi imager, it is convenient to set all the necessary settings like enabling ssh and home wifi , so no monitor and external keyboards will be neccessary.) Then, install PuTTY. 

Finding the ip address of the pi can be done through home router (if DNA automatically assigns it), or ... attach it to separate monitor or keyboard.

Then, when you log in via ssh - first things - 

sudo apt update 

and 

sudo apt upgrade

When repos are set, next thing is installing CUPS (Common UNIX printing system). 

sudo apt-get install cups 

It is not enough tho. HP Laserjet Pro M15a will need hp drivers. This is where hplip can be helpful.

sudo apt-get install hplip

 

After HP drivers are installed, the CUPS needs to be configured so its admin page is available from any computer within my network.

So a CUPS configuration file needs a few modifications.

Editing /etc/cups/cupsd.conf:

 

# Listen on external interfaces for connections

Listen TheIPofMyRaspberryPI:631

Listen /var/run/cups/cups.sock

 

# Show shared printers on the local network.

Browsing On

BrowseOrder allow,deny

BrowseAllow all

BrowseAddress All

 

# Default authentication type, when authentication is required...

DefaultAuthType Basic

 

# Restrict access to the server...

<Location />

  Order allow,deny

  Allow localhost

  Allow All

</Location>

 

# Restrict access to the admin pages...

<Location /admin>

  Order allow,deny

  Allow All

</Location>

 

# Restrict access to configuration files...

<Location /admin/conf>

  AuthType Default

  Require user @SYSTEM

  Order allow,deny

  Allow All

</Location>

 

And now I can restart the service.

sudo service cups restart 

or 

sudo /etc/init.d/cups restart

 

Now lets add Raspberry Pi user to the lpadmin group

sudo usermod -a -G lpadmin your-username

 

Now the printer should be visible in the CUPS configuration page.

Opening the CUPS conf page on the browser - CUPS port is 631 

http://raspberrypiIPaddress:631

Navigate to add a printer. 

Now you should be able to see a printer in the list. 

 

In the end let`s make a Raspberry Pi having a static ip address. I have Mikrotik router. 

Under the section IP find DHCP server and select the Leases tab there. Find your RPi there, click on it, and click on the button "Make Static"

 

Then add printer on your local devices. It should show up.

 

Happy printing!

 

 

Restoring some old P1 computer

 Got a PC from a friend about year ago. Old, with Intel Pentium 100 processor on the MSI mobo MSI MS-5124 

So it had a problem with accessing a HDD. First thought - dead or going to die HDD. Tested and realized HDD working fine, but not correctly recognized. Swapped cables, tried different other HDDs, but when detecting the HDD, some gibberish on screen instead of HDD model. So, it sounds like a mobo or chipset fault. I even swapped the bios. 

Next thing I tried was some IDE controller cards. But those also without success.

And then I tried SCSI controller with SCSI disk. Well, it worked. SCSI disk size is 36.4 Gb. 

Now transferring the system from Win 98 floppy.

Some notes for me and for future.

To make the disk bootable the system must be transferred from floppy to C disk using the command sys C

The sys.com executable is compressed in the cabinet file ebd.cab and must be extracted. Depends of the Win 98 boot disk. I had to extract it. I extracted all files on formatted C disk and then copied sys.com back to A disk and then ran the sys C. Most likely this could be done directly by extracting sys file to A disk, instead of C disk.

Things I used fo avoid floppy nightmare - Gotek floppy emulator (multiple floppy images on usb flash) https://www.gotekemulator.com/

Things to do next - have to get all Win95 floppy disk images, upload it on my flash and install Win95

Because, why not?

Computer hardware abbreviation list

 Starting new job - teaching those young people something about how computers work. In hardware level. This semester will be tough - will have to understand and simultaneously organise the course. 

Moodle, of course, as a platform where to put all tasks, activities, tests, materials. Since university is still setting all of that up for me, had to save one of the activities somewhere. And - why not to share.

So here it is - computer hardware abbreviation list - task to give to studens so they find the definitions of abbreviations so often seen around.

And here the same task in latvian. 

Kā iegūt bulk meteodatus no meteo.lv

 

Darbam ar meteo.lv datiem

Izvēlies staciju, izvēlies parametrus, izvēlies laika intervālu (sākuma un beigu gadus) un dari ar datiem, ko vēlies.

Staciju ID

30000 : Ainaži

30004 : Alūksne

30011 : Bauska

10000120 : Dagda

30018 : Dagda

30021 : Daugavpils

30022 : Dobele

30034 : Gulbene

30036 : Jelgava

30040 : Kalnciems

30046 : Kolka

30048 : Kuldīga

30058 : Lielpeči

30060 : Liepāja

10000118 : Liepāja piekraste

30068 : Madona

30072 : Mērsrags

30081 : Piedruja

30087 : Priekuļi

30080 : Pāvilosta

30099 : Rucava

10000180 : Rēzekne

30092 : Rēzekne

30094 : Rīga

30096 : Rīga - Universitāte

30100 : Rūjiena

30102 : Saldus

30103 : Sigulda

30105 : Skrīveri

30106 : Skulte

30111 : Stende

30104 : Sīļi

30128 : Ventspils

30132 : Vičaki

30141 : Zosēni

30140 : Zīlāni

parametru ID

4514 : Aramkārtas temperatūra 10 cm dziļumā, faktiskā

4515 : Aramkārtas temperatūra 15 cm dziļumā, faktiskā

4516 : Aramkārtas temperatūra 20 cm dziļumā, faktiskā

4513 : Aramkārtas temperatūra 5 cm dziļumā, faktiskā

4167 : Atmosfēras spiediens stacijas līmenī, faktiskais

4457 : Augsnes virsmas stāvoklis

4459 : Augsnes virsmas temperatūra, faktiskā

4464 : Augsnes virsmas temperatūra, stundas maksimālā

4462 : Augsnes virsmas temperatūra, stundas minimālā

4327 : Augšējo mākoņu forma

4001 : Gaisa temperatūra, faktiskā

4008 : Gaisa temperatūra, maksimālā iepriekšējo 3 stundu laikā

4003 : Gaisa temperatūra, minimālā iepriekšējo 3 stundu laikā

4006 : Gaisa temperatūra, stundas maksimālā

4004 : Gaisa temperatūra, stundas minimālā

4002 : Gaisa temperatūra, stundas vidējā

4321 : Kopējais mākoņu daudzums

10307 : Laika apstakļi 1. kods pēdējā 1 stundā;A

10308 : Laika apstakļi 2. kods pēdējā 1 stundā;A

10306 : Laika apstakļi, faktiskie;A

4627 : Laika apstākļi novērojumu termiņā

4676 : Meteoroloģiskā redzamība

9954 : Meteoroloģiskā redzamība faktiskā

4674 : Meteoroloģiskā redzamība, stundas maksimālā

4672 : Meteoroloģiskā redzamība, stundas minimālā

4494 : Minimālā temperatūra zāles augstumā

4323 : Mākoņu augstums

10193 : Mākoņu augstums 1

10194 : Mākoņu augstums 2

10195 : Mākoņu augstums 3

10196 : Mākoņu daudzums 1

10197 : Mākoņu daudzums 2

10198 : Mākoņu daudzums 3

9536 : Nokrišņu daudzums 10 minūšu laika intervālā

4568 : Nokrišņu daudzums starp termiņiem

4570 : Nokrišņu daudzums, stundas summa

4628 : Pagājušie laika apstākļi 1

4629 : Pagājušie laika apstākļi 2

4224 : Piekrastes vēja brāzmas, stundas maksimālās

4317 : Piekrastes vēja virziens, faktiskais

4220 : Piekrastes vēja ātrums, faktiskais

4223 : Piekrastes vēja ātrums, stundas minimālās

4670 : Redzamība jūras virzienā

4080 : Relatīvais mitrums, faktiskais

4084 : Relatīvais mitrums, stundas maksimālais

4082 : Relatīvais mitrums, stundas minimālais

4606 : Saules spīdēšanas ilgums, stundas summa

4342 : Sniega segas biezums

4341 : Sniega segas biezums, stundas vidējais

4343 : Sniega segas biezums, termiņā 18

4344 : Sniega segas seguma pakāpe stacijas apkārtnē

4530 : Summārā radiācija, stundas maksimālā

4528 : Summārā radiācija, stundas minimālā

4527 : Summārā radiācija, stundas vidējā

10188 : Temperatūra zem dabiskās veģetācijas virsmas 0.1 m dziļumā, faktiskā

4495 : Temperatūra zem dabiskās veģetācijas virsmas 0.2 m dziļumā, faktiskā

4496 : Temperatūra zem dabiskās veģetācijas virsmas 0.4 m dziļumā, faktiskā

4497 : Temperatūra zem dabiskās veģetācijas virsmas 0.8 m dziļumā, faktiskā termiņā 12

10253 : Temperatūra zem dabiskās veģetācijas virsmas 1.6 m dziļuma, faktiskā

4499 : Temperatūra zem dabiskās veģetācijas virsmas 1.6 m dziļumā, faktiskā termiņā 12

4500 : Temperatūra zem dabiskās veģetācijas virsmas 3.2 m dziļumā, faktiskā termiņā 12

9880 : Temperatūra zāles augstumā, faktiskā

9883 : Temperatūra zāles augstumā, stundas maksimālā

9881 : Temperatūra zāles augstumā, stundas minimālā

9884 : Temperatūra zāles augstumā, stundas vidējā

10254 : Temperatūras zem dabiskās veģetācijas virsmas 3.2 m dziļuma, faktiskā

10252 : Temperatūta zem dabiskās veģetācijas virsmas 0.8 m dziļuma, faktiskā

4544 : Ultravioletā radiācija, stundas maksimālā

4542 : Ultravioletā radiācija, stundas minimālā

4541 : Ultravioletā radiācija, stundas vidējā

4330 : Vidējo mākoņu forma

4212 : Vēja brāzmas, maksimālās starp termiņiem

4218 : Vēja brāzmas, stundas maksimālās

10208 : Vēja virziens, faktiskais (10 minūšu vidējais)

4313 : Vēja virziens, faktiskais (2 minūšu vidējais)

4211 : Vēja ātrums, faktiskais

4216 : Vēja ātrums, stundas minimālais

4322 : Zemo mākoņu daudzums

4324 : Zemo mākoņu forma

In [1]:

import requests as reqs
from requests.packages.urllib3.exceptions import InsecureRequestWarning
reqs.packages.urllib3.disable_warnings(InsecureRequestWarning)

Tā kā meteo.lv ir https, tad, veidojot zināmu pieprasījumu, ignorēsim zināmu pieprasījumu.

In [2]:

sc="https://www.meteo.lv/josso_security_check"
sec_check=reqs.post(sc)
cookies=sec_check.cookies.get_dict()
cookies["JSESSIONID"]

Out[2]:

'DB28619DC4248262E9EEDFFEA023CFAB'

Lai varētu lejuplādēt datus, nepieciešams cookie ar nosaukumu JSESSIONID. No pārlūka atverot datu meklēšanas lapu, tiek izveidots redirekts uz šo un atpakaļ uz datu meklēšanas lapu.

Tālāk jāsagatavo pieprasījums.

1. Būs vajadzīga adrese url
2. Būs nepieciešamas vismaz dažas lietas iekš header

In [3]:

url="https://www.meteo.lv/meteorologija-datu-meklesana/?"

headers={
"Content-Type":"application/x-www-form-urlencoded",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"Referer": "https://www.meteo.lv/meteorologija-datu-meklesana/?nid=461",
"Cookie": "JSESSIONID="+cookies["JSESSIONID"]
}

Noderīgi būs arī izveidot datu nosaukumu sarakstus. Tie sevī ietver novērojumu staciju nosaukumus un parametru nosaukumus

In [4]:

import json
saraksts=reqs.get("https://www.meteo.lv/klasifikatoru-filtrs/?iBy=station&iStation=&iParameter=4001&pMonitoringType=METEOROLOGY")
saraksts= json.loads(saraksts.text)

Var redzēt, ka klasifikatoru sarakstu var iegūt pēc pieprasījuma, norādot, piemēram, parametra id. Tiks nofiltrētas tās stacijas, kurās tiek veikts šī parametra mērījums.

In [5]:

stacijuSaraksts={}
for each in saraksts["stations"][1:]:
    stacijuSaraksts[each["id"]]=each["name"]

parametruSaraksts={}
for each in saraksts["parameters"][1:]:
    parametruSaraksts[each["id"]]=each["name"]

Tiek izveidotas vārdnīcas, kuras saturēs staciju un parametru nosaukumus - vēlāk būs ērti strādāt ciklā un glabāt failus, to nosaukumos norādot šo info.

Šajā brīdī viss ir sagatavots un var palaist galveno ciklu. Jānorāda stacijas ID, parametra ID, kā arī laika periods - sākuma un beigu gads.

In [7]:

####
stationID=30022
paramID=4001
startYear=2016
endYear=2020
####


for year in range(startYear,endYear+1):
    StartDate="01.01."+str(year)
    EndDate="31.12."+str(year)
    params="iBy=station&nid=461&pMonitoringType=METEOROLOGY&iStation="+str(stationID)+"&iParameter="+str(paramID)+"&iDateFrom="+StartDate+"&iDateTill="+EndDate
    fname=stacijuSaraksts[str(stationID)]+"_" \
    +parametruSaraksts[str(paramID)] + "_" \
    +StartDate+"-" \
    +EndDate+".xls"
    print(fname)
    
    result=reqs.post(url,verify=False,data=params, headers=headers)
    with open(fname, 'wb') as f:
        f.write(result.content)

Dobele_Gaisa temperatūra, faktiskā_01.01.2016-31.12.2016.xls
Dobele_Gaisa temperatūra, faktiskā_01.01.2017-31.12.2017.xls
Dobele_Gaisa temperatūra, faktiskā_01.01.2018-31.12.2018.xls
Dobele_Gaisa temperatūra, faktiskā_01.01.2019-31.12.2019.xls
Dobele_Gaisa temperatūra, faktiskā_01.01.2020-31.12.2020.xls

Darbs ar saglabātajiem failiem

Saglabātos failus būtu vērts apvienot. Lai to izdara pandas.

In [8]:

import pandas as pd

In [12]:

####
stationID=30022
paramID=4001
startYear=2016
endYear=2020
####

df=[]

for enum, year in enumerate(range(startYear,endYear+1)):
    StartDate="01.01."+str(year)
    EndDate="31.12."+str(year)
    fname=stacijuSaraksts[str(stationID)]+"_" \
    +parametruSaraksts[str(paramID)] + "_" \
    +StartDate+"-" \
    +EndDate+".xls"
    #df[enum]=pd.read_excel(fname)
    df.append(pd.read_excel(fname,skiprows=1,parse_dates=["Datums \ Laiks"],index_col=0,dayfirst=True))
    df[enum]["vidējā"]=df[enum].mean(axis=1)
    #df[enum]["summa"]=df[enum].sum(axis=1)
    
    

Ciklā tiek norādīti tie faili, no kuriem vajag apvienot datus, balstoties uz to nosaukumu, kas sastāv no stacijas, parametra un gadiem. Ir izveidots mainīgais df, kas ir masīvs un satur visas pandas tabulas. Pie reizes arī aprēķināta visu rindu vidējā vērtība (šajā piemērā - dienas vidējā temperatūra)

ar pandas.concat() palīdzību šis masīvs tiek apvienots vienā kopējā datu tabulā dataf

In [13]:

dataf=pd.concat(df)

In [14]:

dataf

Out[14]:

	00:00	01:00	02:00	03:00	04:00	05:00	06:00	07:00	08:00	09:00	...	15:00	16:00	17:00	18:00	19:00	20:00	21:00	22:00	23:00	vidējā
Datums \ Laiks
2016-01-01	-14.5	-12.9	-12.5	-11.6	-11.1	-10.8	-11.0	-11.0	-10.5	-10.0	...	-8.9	-9.2	-9.3	-9.1	-9.1	-9.2	-9.4	-9.8	-11.5	-10.183333
2016-01-02	-13.9	-14.6	-15.1	-15.7	-16.0	-16.4	-16.9	-16.9	-17.0	-16.7	...	-14.9	-15.5	-16.9	-16.6	-16.7	-17.0	-17.2	-17.4	-18.6	-16.208333
2016-01-03	-19.6	-19.4	-20.8	-20.4	-20.5	-18.3	-16.8	-14.7	-13.1	-11.5	...	-8.8	-8.3	-8.3	-8.4	-8.4	-8.3	-8.0	-7.4	-7.1	-12.404167
2016-01-04	-6.8	-6.7	-7.3	-7.2	-7.4	-6.5	-6.4	-7.0	-9.2	-10.0	...	-9.7	-9.9	-10.0	-10.2	-10.5	-10.6	-10.9	-11.1	-11.2	-8.983333
2016-01-05	-11.3	-11.4	-11.6	-12.9	-14.0	-14.1	-15.5	-14.5	-13.6	-14.8	...	-13.0	-14.0	-13.1	-13.7	-15.7	-16.9	-17.6	-18.2	-18.9	-14.187500
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
2020-12-27	-1.0	-1.1	-1.6	-0.9	-0.7	-0.6	-0.7	-1.2	-0.7	-0.5	...	-0.7	-0.9	-1.2	-1.2	-1.1	-1.0	-1.4	-1.5	-1.9	-0.891667
2020-12-28	-2.0	-2.3	-2.5	-2.4	-2.0	-2.4	-2.2	-1.7	-1.2	-0.4	...	1.2	1.0	0.6	0.8	1.1	1.4	1.1	1.2	1.4	-0.250000
2020-12-29	1.4	1.6	1.8	1.7	1.9	1.9	2.0	1.8	1.9	1.9	...	1.9	2.0	1.9	1.0	1.0	0.7	1.8	2.4	3.1	1.775000
2020-12-30	0.9	2.7	3.4	3.2	3.3	3.5	3.7	3.5	3.0	2.1	...	2.0	1.4	0.9	0.3	0.1	0.5	0.9	1.9	1.9	2.250000
2020-12-31	1.8	1.8	1.7	1.5	1.5	1.3	1.1	1.4	1.1	1.3	...	1.7	1.6	1.5	0.7	0.4	-0.8	-1.0	-1.0	-1.2	0.941667

1793 rows × 25 columns

In [16]:

dataf=dataf[["vidējā"]]

Saglabāsim tikai visu rindu vidējo vērtību un saglabāsim to excel failā.

In [17]:

fname2=stacijuSaraksts[str(stationID)]+"_" \
    +parametruSaraksts[str(paramID)] + "_" \
    +str(startYear)+"-" \
    +str(endYear)+".xls"
dataf.to_excel(fname2)

In [18]:

fname2

Out[18]:

'Dobele_Gaisa temperatūra, faktiskā_2016-2020.xls'

In [ ]: