2011-12-27

Is user given filename valid?

...and try & catch comes to help.

Code in pastebin - http://pastebin.com/HCsDFtUv

Shortly - the function tries to write file and if success, then it deletes it and returns True. Otherwise it returns False. The function could be extended by trying the writing in specified temporary folder.

Such method allows to avoid from using specific and complex regular expressions.

2011-12-22

dummy csv/xyz writer for latvians

Software (written in Visual Basic 2010, dependent of .NET framework)
http://failiem.lv/down.php?i=ntpoppd&n=xyzCreator.exe


the class code (in VB2010) -
http://pastebin.com/fDL3WUSY

Different countries has their own different regional settings. Sometimes it is very inconvenient when in spreadsheet function the decimal separator is comma (as in Latvia) but for programs you need dot as the decimal separator.
In this simple tool just copy columns, choose settings and save somewhere the file.

2011-12-07

Local max,min and slope count in python list.

Simple function written in python - local maximum and minimum count and slope count in the given 1D array (list). M-shaped function has two maximums (or better say - two ascending/upward numbers in row) and minimums (two descending/downward numbers in row), but Λ shaped- one for both. Sometimes it is needed to know exact count of such maximums or minimums, or both.

2011-11-29

long term monthly mean over 100 years

MS Excel is one of the most popular spreadsheet programs in the world. It is possible to write nice macros in VBA, it is handy and mostly meet a usual user needs. For me it is very convenient data storage type - but when it comes to work with large data I prefer writing macros or using external modifying. And here comes Python, again. Ok, the problem:
I have an excel document with data in one sheet, ~19 columns and daily measurements for 100 year period ~34000 rows. The A column contains date, other 19 columns counting from B - values/measurements. And I need to calculate firstly the long term monthly mean normal values for each column (resulting 12 values for each month) and secondly - percentiles over these 19 columns.
The reading from excel is done using XLRD package and percentile calculating - using scipy stats . These two libraries should be installed additionally to run my class.
The created class is so-so complete with a lot of space for possible improvements; currently it met my needs and I`m ok with that.
The feeling of satisfaction after doing such work manually in excel or creating script is completely different, and in the latter case it`s more positive.
This class is free to use, but if you are using this class, please, make the reference to me and let me know about the project where it has been used.

2011-11-25

for loop differences in vb and python

Where is the difference between this and this code? The function should be the same. But the difference is inside the loop region. In the VB code the loop is written as

For I = 1 To 15

but in python -

for i in range(0, 15):

2011-11-24

Find the treasure

My girlfriend is making  treasure lists all the time in the etsy.com, art shop. I made a treasure hunting game... in the Python, as the third step learning it.
The aim is to get the treasure with no going on the same cells second time. There are three intuitive parts how this game is structured. The first part is - to generate the 2D grid and put the player and the treasure somewhere on it. And here comes the first "but" - as the placements are generated randomly, it is possible, that as game starts, it ends because of possibility that the start locations of the treasure and the player are identical. To avoid it, the coordinates has to be checked, and if they are identical, the player placement is regenerated and again checked in while loop.
The next part is - moves. As the grid is discrete with it`s borders, the player generated moves has to be verified - are they allowed. There are four possible movements - wasd - and each are verified.
And the last part is to verify does the player get the treasure or - are the coordinates not used before.

Check the source code here

second steps programming in python

The key to the success is the idea. And the key to the knowledge is to practice.
Firstly, I started with objects in PHP, then tried to do some work in VB2010 (i hope soon, till the end of the year there will be the result - groundwater and runoff modelling software METQ-UL aka MetCool) and today I tried to refresh my Py-mories... All day long tried to transfer VB code into the Python, but it seems that I`ve some small problems somewhere.
Another thing about Python - I didn`t find a way how to change values inside in method... as example if I have a method in class which does something, like

def f(self,a,b)
     a=a**2
     b=b**b
     return a

and in the same time i need to return value b, I have to make a list or dictionary, like
def f(self,a,b)
     a=a**2
     b=b**b
     D=dict(a=a,b=b)
     return D


In such case it is much more unhandy as it was in VB or Fortran, like

Sub f(ByRef dim a as integer, ByRef dim b as integer)
   ..
End sub

The difference is when such sub (function/subroutine) has to be called, the syntax is simple in the VB/Fortran

a=2
b=3
call f(a,b)
print.debug a,b
//returns 4 and 27

without neccesity to "extract" the modified value from function
unlike the other languages, where it has to be some object or variable which will have the modified values..
in the case of python -

//example when function are inside and are called from the class
a=2
b=3
dictionary = self.f(a,b)
print a,b //return 2,3
a=dictionary['a']
b=dictionary['b']
print a,b //return 4, 27

python lists and dictionaries

This one is specially for those, who mostly are programming in such languages, where the term "associative array" (like PHP) is used.
Python list, as I now understand, is a 1 dimensional unassociative array where each element has its own key mostly with ascending integer.
And the python dictionary is a 1 dimensional associative array where the key for each element can be ... string.

2011-11-10

How to get raster values under the vector points in GRASS GIS

Short video tutorial how to update vector points with raster values which underlies where the given vector are.

Importing XYZ points in the GRASS GIS

Short video tutorial how to create vector point file in the GRASS GIS if you have only x,y and any additional value.


Presentation of Baltic arthesian basin version V1

Today I had an opportunity to present my research results in this presentation. And additionally the local television company LNT took an interview. Of course, emotions are wide enough, as it was my first time to speak on such high class auditory. (After the presentation I loose my focus and... here comes the TV...).

The aim of this project is to develop the human resources which would be able to solve fundamental and practical scientific problems related to groundwater characteristics and resources.

The main result is the integrated mathematical model system which allows quantitatively and qualitatively modelling groundwater

Additionally the impact of climate change on groundwaters has been studied allowing to estimate and predict possible danger of climate change.
And this last topic is where I am personally interested in and working with.
Currently the observations has been summarized and one freely chosen climate model projection used to characterize the future period.

The presentation can be downloaded from here.
And the demonstration can be seen here.

2011-10-22

Point density.

The most simple way how to calculate point density using Quantum GIS.
Simple and fast as ABC.

The text used in the clip can be found in the youtube as a video description.

2011-09-30

Assing an array as object property in Visual Basic 2010

I am not yet completely sure about how correctly this is, but it took a lot of time for me figuring out how to assign an array as a property.
I found out that firstly I had to define an array variable inside a class

Private temperatureArray() As Double

Then I have to make a property, which returns element of the array


    Public ReadOnly Property temperatureVal(ByVal index As Integer) As Double
        Get
            Return temperatureArray(index)
        End Get
    End Property


And finally I have to assign the values to an array inside a function or sub


    Sub getTemp()
        ReDim tempVal(UBound(valuesArray)) 'valuesArray is some predefined array.
        For i = 0 To UBound(valuesArray)
            temperatureVal(i) = valuesArray(i)
        Next i
    End Sub

Simple textfield validating in Visual basic 2010

Simple way how to validate text fields.
If you have text field and you are using "_textChanged do something", then you probably know that it could be not enough. Depending on different specifications (usually such input is for numbers) and writing bad data like characters where should be numbers, it can make errors.
From my little experience, VB and similar program code is working based on "on error do something else".
In the following example I`m doing two things.
In the first I`m validating each keypress allowing just numbers, comma and decimal seperator,


Private Sub step1_KeyPress(ByVal sender As Object, ByVal e As System.Windows.Forms.KeyPressEventArgs) Handles step1.KeyPress
        If (Not Char.IsNumber(e.KeyChar) AndAlso Not ".,-".Contains(e.KeyChar) AndAlso Not e.KeyChar = Microsoft.VisualBasic.Chr(Keys.Back)) Then
            e.Handled = True
        End If
    End Sub

And in the second - catching empty-string error. If someone deletes all content in the textbox, text is changed and passed string is empty, which does no good.


    Private Sub step1_TextChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles step1.TextChanged
        On Error GoTo 11
        If Not (step1.Text = String.Empty) Then
            parametrs.setStep() 'this pass error
        End If
11:
    End Sub

2011-09-17

First/last day of the month

2011.09.17.
Is the given date the first day of the month? Or maybe the last? Whatta silly question... of course not the first or last one! But sometimes it is useful to know or.. better let`s say, return "true" on that kind of questions.
I found that it could be done very easily with creating object from such string, returning parts, making new date value and done some comparing.
Ok, here are the steps-
$dateObj= new DateTime($datestring);
$currentDay=(int) $dateObj->format('d'); //make int not string. let`s get necessary date parts
$currentMonth=(int) $firstDate->format('m');
$currentYear=(int) $firstDate->format('y');

//let`s create another month value returning from the new date, where it is created from the day //before current date
$prevMonth = (int) date("m", mktime(0, 0, 0, $currentMonth, $currentDay-1, $currentYear));


//and finally - comparison
if ($currentMonth>$prevMonth)
{
return "true";  //if previous date is in the previous month, we have the first day of the month
//from date object parts you can make new date here based on your needs through mktime.
}
else
{
return "false";
}

and now - if we need to know is the date is the last day in the month.
$nextMonth = (int) date("m", mktime(0, 0, 0, $currentMonth, $currentDay+1, $currentYear));
if ($currentMonth<$nextMonth)

See the changes?

Avoiding injections

Very fast and useful way, how to avoid different injections in websites - if you know what type of variable can be.
For example - if you have any ID`s or other values which can be only integers - in the script you can always be sure that after such validation it will always be an integer just writing (int) before the variable - like -
$variable = (int) $_GET['variable'];
 Another example - if you have variable which could always be date - you could pass it as a string and build a date object from it
$date=$_GET['date'];
$dateObj = new DateTime($date);
//if you need to get back to string in any case
$dateString = $date->format("Y-m-d") //or other format
Dealing with the float type it is very similar like integer type. But it is always necessary to remember - what is the decimal seperator of the host and decimal seperator in your country. In my case, I have to remember, that the host`s decimal seperator is dot, but usually people in Latvia are using comma. So, I have to parse the given "float string", and replace comma with dot, if there`s any.

$value=(float)str_replace(",",".",$value);

Linear interpolation. Extended version.

I already wrote about the linear interpolation and related problems which were successfully solved. However, full convenience requires some monthly statistical capacities. Usually the data series, including interpolated ones, does not start and end with first and last day of the month, respectively and thus the first thing is to cut off the first and last incomplete month measurements. And only then it is possible to split the data series by month. Splitting occures in the loop splitting the corresponding date and getting the month and comparing it with the previously splitted month value. If the measurement is in the same month, the new two dimensional array is created with the date (year and month) in the first dimension, and (measured including interpolated) value in the second. And if the current month value differs from the detected month value in previous cycle, it is time to change the first - date dimension value to the next one. After creating such array, it is time to make some statistics for each of inner measurement array.
You can try the linear interpolator here and see the source code (not yet OOP, but i hope soon) - here.

2011-08-31

google maps and proj4js. movable marker

Another script which combines proj4js and google maps together. This one gets coordinates from one system (EPSG:25884) which are generated in input boxes (and if there is none or wrong format, gives default values), converts to wgs84 lat/long and sets a movable marker on the google maps. As the user is moving marker, it defines the new center of the map after the user releases it and the new coordinates has been calculated back from lat/long to epsg:25884 and putted in the text box.

2011-08-30

Linear interpolator script

Mostly measured data in databases are defined in exact time or date. But sometimes it is necessary to interpolate between different dates because of skipped measurements. For example, in earlier times, groundwater level measurements were made "by hand" after every three to seven days, depending of site and monitoring type. But nowadays groundwater has been monitored using different loggers, which are called - divers. These little gadgets gives possibility to get daily data.


The problem is that working with daily measurements and not taking care about skipped days, statistical weights could be different from the case, when in these skipped days the measurements are  interpolated.

I made a little script, where measurements and their corresponding dates are taken from mySQL database using PHP; and in the database there could be only measured data - this script interpolates the missing values.

2011-08-27

Yesterday I had The possibility to fly as a passenger in motor-deltaplane.
The feeling is great. All the time. Taking off, curves with feeling of falling, like not moving forward but.. rotating downward. The feeling was so strong and impressive, that I have almost forgotten the scenery. But maybe it is becouse of increased epinephrine release.


From now on I am looking for sponsorship and donations which could cover the costs of getting the PPL (personal pilot licence).

If my dream will come true in the future, I promise to support the science with my gained experience in every possible way.

2011-08-22

Ryanair B738, flight FR-8085 incident

Landing gear problem.

Simon Hradecky writes in the Aviation Herald:

A Ryanair Boeing 737-800, registration EI-DWT performing flight FR-8085 from Brussels Charleroi (Belgium) to Riga (Latvia), was on approach to Riga when the crew reported a gear unsafe indication, performed a low approach which confirmed one of the main gear struts was not down. The aircraft climbed back to 6000 feet and entered a holding to trouble shoot the problem. About 45 minutes after aborting the first approach the crew managed to lower and lock all gear and landed safely on Riga's runway 18.
Riga Airport reported that there were already considerations to have the aircraft ditch in the Gulf of Riga just north of the aerodrome.
Latvia's Directorate General of Civil Aviation confirmed the aircraft had a problem with the landing gear. The aircraft subsequently entered a holding area to work the problem, then performed another overflight (according to radar data at 2500 feet MSL) for another visual check by experts from the ground which confirmed all gear had reached the fully down position and commenced a safe landing immediately thereafter.

Media (www.apollo.lv) reports unusual pilot activities asking for passengers to pray God.

Digging in the internet it seems that mostly there are no fatalities in landing gear accidents. So, the most interesting question here is why there were some profesional thoughts about ditching in the sea...

And here is that baby




This is my favorite comment from avherald:

Combine the vivid imagination and crap english knowledge of people flying once in 2-3 years and that is what you get when you interview them. You could also tell that a moon landing was considered.. :)

1) Final announcement about landing was made on approach which is over the gulf - here you have ditching.

2) At some moment passengers were asked to assume brace position. To some "brace" may sound like "pray". The position itself sparks some associations - here's the praying part. 

2011-08-18

Puma vs Firefox

No. This is not THAT Puma brand. This is European funded science project.

Doesn`t it look familiar..? orange circle with blue inside


2011-06-03

2011-05-25

Since QGIS has simple and poverful GRASS GIS implementation, I found that more often I`m using this Grass version, not standalone Grass GIS.
There has been some needs for my friends doing very basic and simple things, but as it takes longer if someone don`t know QGIS and Grass well, I made some tutorials.

This one explains how to define coordinate system for project for correct distance and area measurements, and correctly add scalebar.

In this next video the DEM is loaded in Grass GIS database from QGIS and visualised through NVIZ. All under windows xp.





The next video shows how to georeference an image file. I`m using very dummy image file as example... ;)



The next video shows how to rasterize vector contour lines, how to interpolate rasterized contour lines and very basic mapcalc stuff - how to get difference between two different DEM`s.



This video shows a bit complicated raster analysis example. There is a dummy JPG raster file where each cell should contain values 0 (black) and 255 (white). But because it`s compressed and it`s JPG, there are cells between defined values. With simple mapcalc usage it`s possible to define - are these cells could be counted as black or white.
of course, it`s possible to do it just writing script in shell, but Grass allows create these scripts in very user friendly way.



But sometimes there are needs to manipulate with LIDAR data. And here LAStools are very handy. LAStools could be downloaded from here. But as they are command line tools, the easiest way is to write BAT`ch file (in windows).
So here is example - how to manage it.

2011-03-09

Cannot run commands from command prompt?

Playing around with GRASS and Python, accidentally I lost environment variable resulting that some commands, like ipconfig and ping, from command promt were not running and the error was called.
Windows system32 directory should be left as one of the PATH variables.
PATH=D:\GRASS-64\etc;D:\GRASS-64\etc\python;D:\GRASS-64\lib;D:\GRASS-64\bin;D:\GRASS-64\extralib;D:\GRASS-64\msys\bin;D:\Python27;D:\Windows\System32



2011-03-02

daily data

PHP mysql database code for wells. daily results for all wells.
Sometimes there`s a need for daily data, but in database there are records only about those days, when measurements were made.
In given example, it is possible to set the start and end days, and multiple measurement points a.k.a. wells.

2011-02-25

HeidiSQL

Very nice program to access remote SQL server.
Also, I`ve learned some neat things about MySql functions and extended use.
For example, here is some code
What does it do?
It`s looking for all similar months and returns an average value from all values on given month.
In the inner "select" I`m selecting all columns and I`m adding additional column, where the each month (meneshi) is added in each row using "where"
And then in the outer query I`m gruping values by newly created month column and calling an average function. It returns all average values for each group - month.
And lastly I`m ordering results by month. Easy as pie, actually.

Only thing I need to change is "ns" in third line from end of the script.
Next challenge would be to make average results for all ns.
select avg(lim), datums, meneshi
from
( select ns, lim, datums,
CASE
WHEN datums like '19__-01-__%' THEN '01'
WHEN datums like '19__-02-__%' THEN '02'
WHEN datums like '19__-03-__%' THEN '03'
WHEN datums like '19__-04-__%' THEN '04'
WHEN datums like '19__-05-__%' THEN '05'
WHEN datums like '19__-06-__%' THEN '06'
WHEN datums like '19__-07-__%' THEN '07'
WHEN datums like '19__-08-__%' THEN '08'
WHEN datums like '19__-09-__%' THEN '09'
WHEN datums like '19__-10-__%' THEN '10'
WHEN datums like '19__-11-__%' THEN '11'
WHEN datums like '19__-12-__%' THEN '12' END as meneshi
from hmetlimeni where ns=9664 ) as tble
group by meneshi
order by meneshi

2011-01-30

Python scripting under GRASS GIS

It was few months ago when I using GRASS last time. Since then there was few things that came in my mind about scripting. I discovered, that it was very easy building and launching bash scripts under windows environment, but I never tried Python.
Today, just accidentally, I was starting to think about it again, and discover, that it is not as easy as bashing, but very close.
The first thing was - install Python. In my case I was using Python 2.7.
The second - to set neccessary environmental variables.
And here comes where attention is needed.
In GRASS GIS homepage there are some samples using environmental variables, but it is possible, that user`s machine/computer is differentely configured.

So, here they are:


GISBASE= C:\GRASS-64
GISRC= C:\Documents and Settings\user\.grassrc6
LD_LIBRARY_PATH= C:\GRASS-64\lib
PATH= C:\GRASS-64\etc;C:\GRASS-64\etc\python;C:\GRASS-64\lib;C:\GRASS-64\bin;C:\GRASS-64\extralib;C:\GRASS-64\msys\bin;C:\Python26;
PYTHONLIB= C:\Python26
PYTHONPATH= C:\GRASS-64\etc\python
GRASS_SH= C:\GRASS-64\msys\bin\sh.exe


1st - Drive letter. In my case it`s D:\
2nd - user. Usually it is the name of user.
3rd - Where the GRASS is stored
4th - Where the Python is stored
5th - What version of Python is used.

That`s it.

2011-01-19

Getting categories out of blogspot/blogger feeds

Previous post about coding was about syndicating images in PicasaWeb with JonDesign`s Javascript gallery which uses mootools. Next task was mixing blogspot/blogger feeds in standalone web page. And again using easyXML in PHP5 it is easy as pie. In this script the interesting part is learning what the "child" is. If there is one big main structure, where similar smaller structures are built in, then these structures are like children to this main one. And in blogspot/blogger feeds there are possible to get tags or categories for each post. Additionaly, the term of the category is as attribute name inside the feed`s tag. It`s necessary to use attributes()->term, where term is attribute which has different possible names. And as these attributes could be more than one in each post or entry, then there it is where child is born. A new, child simpleXML object is created within this entry, and, correctly parsing, it is possible to get multiple items in each post. "foreach" entry categories loop inside of "foreach" entries loop.
And here is code to see it how it looks like and another DEMO link how it works.

Thus it is possible to get new blogspot/blogger feeds which are based on given tag/category, when building the hyperlink just add "/-/$category_name" in the end of blogger/blogspot full feeds link.

by the way the full blogger/blogspot feeds link is
"http://blogspotuser.blogspot.com/feeds/posts/default"
And modified link should look like

http://blogspotuser.blogspot.com/feeds/posts/default/-/category

Publication

In SPIE proceedings, about mesopic vision, its characteristics.
Last year, along masters studies in geography, I studied Ph.D in University of Latvia, Physics, mechanics and astronomics program, about psychophysics. And only thing I`d done was I, my proffesor and another collegue, we were going to this lab in France, Clermont-Ferrand to make psychophysical related experiments in fog chamber.

2011-01-17

Javascript + mootools + picasa

At last. Not so easy task finished. Finally combining Google provided Picasa online XML RSS feeds with PHP through EasyXML options with JonDesign`s Smooth Gallery which works with mootools. And here all hard work explained in simple words.
Problem - need nice web gallery. Building myself - no thanx. Using ready product - sounds good.
Problem needs. Just one need. Compatability with image servers like flickr, picasa, photobucket, etc.
Solution is combination of Javascript based web gallery with correct feed parsing.
And personally for me it includes learning new things for me, like EasyXML in php5.
Ok, how to combine these two?
1) Download JonDesign`s with mootools here.
2)Get your rss feed and learn google api here
3)And look for source here

2011-01-14

mysql queries..

SELECT DISTINCT ns
FROM (

SELECT ns, MAX( `sloj` ) m
FROM `w2`
GROUP BY ns
)foo
WHERE m <2041
ORDER BY `foo`.`ns` ASC

and another


select id, max(vērtība) m from tabula group by id having m <= 5