SjASMPlus от z00m

[Ped7g]

Hello everyone, let me introduce myself... and apologize for writing in English, but my Russian knowledge is extremely limited (like 10 words), so I'm reading your forum mostly through google translator, which may be sometimes a bit inaccurate, bear with me.

I'm Ped ("7 Gods" demo group, active on ZX scene around 1992-1997), and currently I happened to be contributor to z00m's branch of sjasmplus.

I will try to help you with any issues or explain what is happening with the project, and where I'm heading with it.

First things first, this one is probably most important to explain:

Печально что эти парни взяли за основу древнейший код, незаметив "основную", более новую ветку, которую правил @Vitamin и позже я. Во многом заново наступают на/правят те же самые грабли, древнейший код.

- - - Добавлено - - -

Да вот действительно, вспоминается анекдот про "ты хотя бы лотерейный билет купи". Уж сколько раз я предлагал писать запросы/багрепорты в issues. Один @asve79 что-то пишет иногда, а больше никому ничего не нужно судя по всему.

А это зачем нужно? Аласмовские сорцы токенизированные же? Чем их редактировать на PC?

This is very unlucky coincidence, and I have no idea why the guys originally followed the older branch (I think the first commit from z00m is around May 2016, so he was either unaware of the new progress, or he had things on his disk for many month, or .. I don't know). I personally ran into this branch by following ZX Spectrun Next group, where somebody forked it and added ZXN extra instructions to it. After doing review of this change set and fixing it a bit more, I was contacted by some .sk demo sceners (I know some of them personally - back from the ~1995 era, when 7 Gods did write few ZX demos) participating on this project, and asked to become a contributor.

At this point I did somewhat deeper review of the "current" z00m's version of sjasmplus, and found out it was basically full of bugs, and almost nothing was working as documented in the (well written, thanks to Aprisobal) documentation. So instead of following my original plans (more about that at end of this message), I started just fixing bugs... somewhere half into it (around v1.12.0) I found out about your sjasmplus/sjasmplus branch on github, wasn't aware of it before.

So I did check how it looks, and unfortunately the code diverted considerably in the meantime, so there's no simple way to sync between the source code directly, but I was able to at least run some of your tests and fix few more bugs, thank you.

In the end I'm afraid it will be like this, two separate branches, two separate developments, and maybe we can share at least some ideas, tests and designs (but I don't feel any huge urge to sync+merge into one branch, having two versions of almost the same thing may be sort of redundant, but it already happened, so the "wasted time" is already gone, merging now would take lot of extra effort, which I don't feel currently like wanting to work on that).

My part on this mistake is insufficient research before jumping into the project, I was not even aware of your and Vitamin's hard work, I'm sorry for this mistake on my side.

about my plans with z00m's sjasmplus brach (and what happened so far since I jumped into it):

• the bugfixing phase is now mostly finished, i.e. v1.13.0 should mostly behave in sync with Documentation (need further reports of issues, especially problems with real projects are very welcome and would be sort of refreshing, because most of my work was done by theoretical reasoning, which sometimes doesn't reflect real world problems very well)
• ZX Spectrum Next device support is simple, but finished (I may revisit it after working on my Next projects, or if somebody reports particular problem)
• when I was fixing bugs or adding new things, I was trying hard to stay backward compatible as much as possible, but some of my changes may have broken old sources. If such change is documented in What's new chapter, it means I had some serious reasons to prefer the change over compatibility and it is not an accident. If you are affected by such change, please consider if you may "fix" your source in the new way.
• if you can't find any change describing things which are broken for you, open an Issue and I will check what is going on, it may be new bug introduced by my code modifications (you can actually open even Issues for thing in previous point, so I get better feedback from real world, how sjasmplus is being used, and I may try then to convince you that my reasons are good enough to require that change, or I may think more about how to make it work in future)
• PLAN: add more power (adding some operators for labels, improving listing files, etc) to work with large device memory in more convenient way (my personal interest is ZX Spectrum Next, but most of these things do apply already for ZX128 and other extended Spectrums)
• PLAN: syntax option to switch-on more strict syntax parsing (probably configurable a lot, because when I tried to collect personal preferences of current sjasmplus contributors and friends, each of us has different expectations what the assembler should report as error/warning), as most of my bugs while writing ZX Next test suite few months back were coming from relaxed syntax, when I wrote something into source and didn't realize it will be assembled in other way than I expect. ( `ld a,(flag1|flag2)` being probably most painful for me, hitting me like three times at least, hehe)
• Add more automated tests to cover ideally all functionality of sjasmplus, to find out more bugs, and prevent simple regressions when changing the code. (currently I have almost 150 tests, covering probably about 1/3 to 1/2 of sjasmplus functionality (as documented))

I will add (at least) two more replies, one trying to cover things reported in previous posts, and finally the v1.13.0 announcement... (to be continued) .. and maybe post some design ideas in greater detail, so you can share your opinions about the planned changes, and help to asses which change will actually help you with Speccy projects, or what is missing for you, to make it better.

- - - Updated - - -

		ld      a, '\\'

The parser was fixed to be in sync with the old documentation, and only strings in "quotes" support escaping. Your '\\' is now in new version really value 0x5C5C. To get value 0x5C it is enough to write '\' (or "\\"). The only "escaped" thing in single apostrophe is the apostrophe itself, and to "escape" it you have to put two consecutive of them, for example four apostrophes in source '''' form value 0x27 (ASCII apostrophe).

While this is unfortunately breaking some older sources, it's not my idea, I just fixed the sjasmplus to follow the old documentation (and I believe it's trivial to update your sources, if you want to use newer version actively). (my idea was to add double-apostrophe in apostrophes parsing)

Also I have seen somewhere report of ` ex af,af' : ...some code : ex af,af' ` not working properly, which I'm pretty sure should work OK in recent versions (probably even since v1.11.0, or when I did fix the parser).

все повторы по старинке через DUP/REPT
в Storm'e было после .4 - удобнее.

I don't understand this one, sjasmplus did already support something like

label .5 add a,a

- which will produce five times repeated instruction ADD. Maybe I misunderstood it? (for larger block you must do DUP 5 : ... block : EDUP, but this seems to me like trifle.

с системными переменными та же шляпа со 128

Sysvars are "correct" pretty much only in ZX48 device case, and I have no detailed knowledge of 128 or other clones (and sysvars for ZX Next are kinda too many and too spread into different banks, and of little interest to me, as my projects take full control over machine and don't need any sysvars at all, but I may work on these in case I will run into some problem with particular project).

If you can provide particular project which needs different sysvars pre-filed (and in which bank), and you can describe what changes are required to make sjasmplus better, add Issue on github with detailed description of changes needed, I will gladly take a look on it, but my 128+ Speccy knowledge is very limited, as I met in real life in my Speccy-era only with 48k machines and various Didaktik clones, so I know very little about other machines and I need detailed description of everything.

не понравилось, что при компиляции вылезает сообщение об include/incbin

check --msg new options, it was annoying to me too, with level set to warnings or errors only the incbin is silent.

***
About support of different assemblers (and their syntax) - I have very little experience with other ZX assemblers ... ok, not very little, just "none". Back in nineties I was either using BASIC DATA with POKE and translating machine code on paper, or later I was using MRS on the device itself. Also I generally dislike the idea of universal assembler which has so relaxed syntax, that it can parse any source, there are then too many ambiguities (I already tried to read documentation of Pasmo, nsasm and Zeus, and there's no sane way to support all of that syntax in one project, as they clash with sjasmplus syntax and between them - what was reasonably compatible like DG/DH/HEX, was added, but generally in my opinion you are expected to pick assembler for particular source and adjust to syntax of that assembler .. it's often just few lines to modify, for example I managed to modify Manic Miner commented source (for Zeus) to assemble with sjasmplus with only few modifications (replacing the zeus keyboard macro with constants being the most invasive change needed)).

I'm still trying from time to time to assemble Nirvana+ with sjasmplus, so far every try resulted into new bugfixing round of macros and similar stuff, I will probably try again with v1.13.0 in following weeks, this one is really tricky...

***
About LUA: I'm not judging its inclusion, although I'm trying to make sjasmplus itself powerful enough to not need Lua at all (normally), but having it available for extra cases... I don't see problem with that, or any benefit gained by removal of it.

About upgrading particular version of Lua - I will try to take a look and see if it's possible to update it, but it better should compile also on mac and even in windows, so it will require some time, as I have only Linux machines available to myself and I can't test these things easily (almost every release preparation I get reports of z00m about breaking the windows or mac build, and those are usually just "innocent" small changes). But I like to have all libraries up to date in my projects, so this is of some medium priority.

I was even making fun of JS to M.Borik (who works with JS a lot, I personally can't stand it), and he did show me actually very small libraries which may provide support of JS in similar way how Lua is supported now, so I'm also considering about adding this. I will probably never use it personally, but some people may...

части команд которые как бы нужны...

You can check project's TODO.txt to see what I am thinking about, and what may be added somewhere in the future (as time will permit, as I already spend ridiculous amount of time on this project, and I'm somewhat missing that time in my life and professional work now, so sadly, I will have to slow down a bit with sjasmplus).

You can always open Issues with particular proposals, what would help you with writing Speccy projects (or even posting it here, but opening issue on github will make it harder to get lost over time, at worst it will be just ignored). I'm especially interested into real problems of particular real projects, as I have already lot of feedback from "theory" people (working on new tools, debuggers, or just trying to push limits of sjasmplus), but almost no feedback from real projects.

- - - Updated - - -

(oh, the new posts gets merged automatically, which will make this a very long post about many different things, but you are probably used to it on this forum)

Just recently the v1.13.0 was released: https://github.com/z00m128/sjasmplus...es/tag/v1.13.0

Changelog (including version v1.12.0 missing in this thread):

v1.12.0:
- Fixed parsing of expressions starting with string literal
- Fixed listing of DS directive with negative values, added value check warning
- Fixed possible wrong indexing of some arrays internally
- Fixed parsing of single-word instructions in colon-packed-no-space macros
- Making macro arguments substitution a bit more aggressive, to work also with
DEFINE/IFDEF/IFNDEF/DEFARRAY directives inside macro (this gives the coder
more macro power and freedom, but the error reporting may get lot more confused)
- Refactoring substring substitution for macro-arguments and defines, now it
should work always. To prohibit some macro/define to substitute into middle
of the string, start its name with underscore, like "_VERSION".
- THIS MAY BREAK SOME OLDER SOURCES, SORRY. Also the rules for substitutions
will be in the future further modified and documented, to make them more
intuitive and predictable, so there may be more breakage even later.
- Adding fake instructions break and exit for CSpect emulator (--zxnext=cspect)

v1.13.0
- [may break old sources] DEVICE: each assembling pass resets also
"device". To work with "device" memory (savesna/savetap/...) you
must select the device (and slot and pages) before producing machine
code which you want to work with (SAVESNA/SAVETAP/...).
If only single DEVICE is used in whole source batch, then the setting
is "global" and will be applied to all lines of source (in 2nd+ pass).
- [may break old sources] ZXSPECTRUM128 based devices map into slots
by default banks {7, 5, 2, 0}. (was {0, 5, 2, 7} in older versions)
- [may break old sources] ZXSPECTRUM128 based devices have sysvars and
stack set up as in "USR 0" mode (ZXSPECTRUM48 system variables and
default stack content).
- [may break old sources] MAP+FIELD directives removed (STRUCT is better and working)
- MMU directive (fusing SLOT + PAGE and extending them)
- SAVEDEV directive (similar to SAVEBIN)
- SAVENEX directive (for ZX Spectrum Next)
- INCBIN: support for negative offset/length values, support for MMU wrapping
- INCBIN: support for file chunks of 64+ki size (usable with MMU)
- Fixed: INCTRD offset, binary STDOUT on windows, SAVETRD/SAVEHOB filenames
- Fixed: LUA used inside macros, LUA get_word, LUA error reporting
- parser: added C++(like) numeric literals
- ZXSPECTRUMNEXT device added
- refactoring of label/define implementation = less memory leaks, more correct
"label.mem_page" values, "Unreal" labels dump is more correct too
- docs: now the CSS file is actually used, and default style modified a bit
- new MACRO examples, syntax-highlight file for KDE5 editors (Kate)

Documentation http://z00m128.github.io/sjasmplus/documentation.html (or in the package/cloned source).

[Advertiser]

[krt17]

Хорошо конечно что стараются, у меня лично на эту ветку были надежды до этого:

- [may break old sources] MAP+FIELD directives removed (STRUCT is better and working)

Структуры конечно хорошо, но теперь придется прибавлять <char>. там где это совсем не нужно и делать структуру там где она совсем не нужна. Да можно equ понатыкать, вот удобство так удобство считать вручную. Можно еще disp 0 сделать. Но нафига все эти извраты?
Жаль конечно.

[Ped7g]

...но теперь придется прибавлять <char>...
...Но нафига все эти извраты?

1) not sure what is your issue, some example? (is it about doing few more keystrokes while writing source? That can be it, as that is usually very low priority for me, as that is usually very small part of SW development - compared to reading and debugging)

2) why changes? Because v1.10.4 is too bugged and it's lacking some functions I need for my own projects.

If you by some luck managed to have working source in older version - my respect! - you can still use the older binary to assemble that particular old source, if you don't want to update it.

For new projects you can decide yourself, but I had huge issues with v1.10.4 when I tried to use it somewhat more, beyond trivial instruction assembling. More than half of the examples from documentation didn't work as it was described, and some features of sjasmplus were not described in documentation at all, so I was not even sure if those were bugs or secret features.

But in the end it is very simple. I need assembler for my own projects, and my idea of good assembler is something reliable and predictable - so I don't have to babysit the assembler itself checking the resulting machine code, and I can focus on my programming and debugging my bugs. V1.10.4 was very far from this goal, unfortunately. Originally I did want just to add Next instructions and few more small changes, but then I did read through the source of it, and there was some kind of bug and inaccuracy almost everywhere. So instead of few changes I spend months (!) just fixing it.

I still put backward compatibility as very high priority item, and I believe, considering I made 300+ commits already, I broke minimal amount of old things. But at some points I had some reasons to believe the change is better option.

In case of MAP removal it was mostly it's poor implementation and lack of documentation (and I still believe STRUCT is as good), so instead of spending another week fixing MAP, I removed it (I was keeping it "as is" as long as it was possible, but due to it's implementation it was getting into my way too much lately, clashing with planned changes, and fixing it was too much of work).

Sorry. Still thank you for this feedback, I'm actually surprised anyone was using MAP in their projects, when I was asking in the cz/sk community, nobody had it (but each of us is very different, some prefer EQU and manual adjusting of everything, some write code generators in JavaScript and I did want functional STRUCT producing correct machine code...). So it's good to know this change was more problematic then I expected.

[Bedazzle]

1) not sure what is your issue, some example?

Downloaded sjasmplus v1.13.0 from git, unpacked.
Trying to start it from cmd - just hangs, doesnät matter if I start it with asm source as parameter, or without parameters.
Win 7 home.

[Ped7g]

Downloaded sjasmplus v1.13.0 from git, unpacked.
Trying to start it from cmd - just hangs, doesnät matter if I start it with asm source as parameter, or without parameters.
Win 7 home.

Ouch. Can you please try also some older versions like v1.12.0 and older? Where the issue did start?
(just running the exe from command line should print out the help page)

Or is here somebody else who has Win 7 and can try it? (It is tested by z00m in "Win 7 pro 32b", that should work)

Can you maybe try also decompress the executable (it's compressed with upx: https://upx.github.io/ ), if it still hangs?

BTW, do you have some antivirus? Doesn't it interfere with it, check maybe some logs or "quarantine" or how these things works (I haven't seen windows for like 12 or 13 years, so I'm not sure how to debug these problems in best way)?

[Bedazzle]

Can you maybe try also decompress the executable

Decompressed, and it starts ok.

BTW, do you have some antivirus? Doesn't it interfere with it

Yes, Avast.
Tried again original compressed exe, and it showed sandbox. Yesterday it didn't jumped.
So seems it is ok for now.

But.
Tried to compile source, and



And same source with another branch



I think it was already reported for another branch, and problem was with includes inside include file:

core.a80 file:
....
INCLUDE "known/_7EC7-7F5F___print_situation.a80"
...


_7EC7-7F5F___print_situation.a80 file:
...
INCLUDE "txt/_7ED5-7EDC___situation.a80"
...

[krt17]

But.

По русски написали, если используешь ' кавычки будь готов что экраны не нужны. Это нормально и правильно, и меня кстати пару раз бесило что раньше было не так.
Про MAP согласен что используется редко. Понадобилось всего 2 раза, просто второй раз совсем недавно, поэтому и заворчал.
Удобно использовать для задания общего массива переменных с адресацией через индексный регистр.

[Ped7g]

I have no idea from that post, what is wrong, I see the FREE RAM output is different, but I don't know this project (are the sources available somewhere or can you create some small example just demonstrating the problem?).

Generally include inside include should be NOT a problem, but maybe there's some bug I am not aware of. If you will manage to help it identify, it would be very nice, thank you.

(the changes in code [I am aware of] were about include search paths, the "" paths start search in current source directory first, while <> paths go through provided include paths first and ends in current source dir, just as the docs specs says.

But in your case, unless you have two different files "txt/_7ED5-7EDC___situation.a80" in different folders, this shouldn't affect your code in any way, because either the correct file is found and included+assembled, or error about opening file would show ... anything else about include-in-include shouldn't be of any special significance, and the assembling state should be pretty much identical as if one big source file was provided, so there's some bug somewhere

There were some changes about auto-wrap of addresses, which affect behaviour around 0x10000 address, maybe that's affecting it?)

(Also I see difference in "compiled lines", but that calculation was slightly modified recently, so this is not conclusive, still same amount of lines may have been assembled by both assemblers, it's just the 1.13.0 does count it differently in some cases, probably around multiple instructions on same line, etc.. so this one doesn't bother me, but the 0x10000 vs 0x0000 is worth attention of course)

(the warning about 0x5C5C is very likely the '\\' thing and as there *was* warning, the final code is identical, would be different in case of no warning.. like ld hl,'\\' is now ld hl,0x5C5C but the warning says that the value was truncated, so only 5C was used of it .... hmm... maybe I should add extra warning about backslash used inside apostrophes for a version or two, so users can check all their source and adjust to quotes or remove escaping backslash?)

[mastermind]

Хорошо конечно что стараются, у меня лично на эту ветку были надежды до этого:

Структуры конечно хорошо, но теперь придется прибавлять <char>. там где это совсем не нужно и делать структуру там где она совсем не нужна. Да можно equ понатыкать, вот удобство так удобство считать вручную. Можно еще disp 0 сделать. Но нафига все эти извраты?
Жаль конечно.

MAP/FIELD были выпилены из "основной" ветки тоже давным давно, см. https://zx-pk.ru/threads/447-sjasmpl...=1#post1002128 (в той же теме далее есть пример как с помощью макросов можно делать то же самое)

MAP/FIELD were removed from the "main" branch years ago as well, see https://zx-pk.ru/threads/447-sjasmpl...=1#post1002128 (in one of the following messages there is an example of how macros can be used to do the same, at least for some use cases)