SjASMPlus от z00m

[Ped7g]

About LUA approach: if it's "complete" block of code, you can extract the if/def logic into lua, emitting only instructions
I guess I have some idea why it fails like this, and I think I was already fixing similar case for something else? Can't remember... I may take a look later if it's possible to make similar LUA code generator work, your idea seems valid to me.

About defarray: I really don't want to, that feels like can of worms being opened ... defines feel to me more like C preprocessor than actual script language
when you write "the_array[1] = 4", the line is substituted very early right after reading buffered input into "3 = 4", then are the directives/instructions being parsed. What you propose would require to parse this array-assignment directive ahead of substitution, adding like whole new step in the compiling "pipeline". Maybe it would work well (or maybe it would backfire later), but it really feels like changing defines into something what they are not.

About memory approach: {0} = 3 is `ORG 0 : DW 3`, so this part is not a problem, but the `IF` part is problem, because pass1 and pass2 will see {$4000} as 0, only in pass3 there will be real value... Thinking what you did want to achieve originally, this will not work either.

As long as only fixed control variables are involved like "aaa", you can do that in the asm like:

aaa = 3
bbb = $c0
   dup 10
     if aaa = 5 : db $ff : endif
     db bbb+aaa
aaa = aaa + 1
bbb = bbb -  $10
  edup

But if you want "array" of control variables, then I'm afraid currently you must elevate *all* the logic into Lua, and write full code generator in Lua, can't think of good hack how to do that in asm/define way.

I will take a look if there's a way to fix IF/ENDIF emitted by _pc from Lua script ... that sounds like something what "should" work, if you don't know the sjasmplus internals, and you just follow the documentation, so fixing this would be good. But it may be too difficult.

I believe you can actually make that work if you will emit whole IF->ENDIF block into _pc/_pl, i.e. _pc("if aaa = 0 : db 1,2,3,4 : endif") -> that MAY work?
But I'm not sure if it is enough for you to resolve all your issues.

EDIT: the colons will actually break _pc, right? sorry, I don't have time to try it now, I have really urgent work project upon me, so sorry if my advice is broken. Will take second look later, when I will be more free from work.

- - - Updated - - -

https://github.com/z00m128/sjasmplus...es/tag/v1.13.2

v1.13.2 changelog:
- OPT has now also "listoff" and "liston" commands
- added to --syntax option: case insensitive instructions, "bBw" added/changed
- new macro examples, minor fixes and refactoring
- SAVETRD implementation refactored (more checks, less bugs, "replace" file feature)
- operators "{address}" and "{b address}" are now official and documented

Documentation http://z00m128.github.io/sjasmplus/documentation.html (or in the package/cloned source).

[Advertiser]

[NEO SPECTRUMAN]

I believe you can actually make that work if you will emit whole IF->ENDIF block into _pc/_pl, i.e. _pc("if aaa = 0 : db 1,2,3,4 : endif") -> that MAY work?
But I'm not sure if it is enough for you to resolve all your issues.

нет это не работает
да и мне нужно будет вставить много кода между if endif

		for all_addr_cnt1 = op_jptabs_haddr_start,op_jptabs_haddr_end,1 do
		print (all_addr_cnt1)
		temp1 = string.gsub("if op_addr_xxx = 0 : nop : endif","xxx",string.format("%x",all_addr_cnt1))
		_pc(temp1)
		end

SjASMPlus Z80 Cross-Assembler v1.13.2 (https://github.com/z00m128/sjasmplus)
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
Pass 1 complete (0 errors)
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
Pass 2 complete (0 errors)
32
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
33
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
34
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
35
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
36
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
37
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
38
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
39
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
40
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
41
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
42
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
43
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
44
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
45
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
46
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
47
Test.asm(33): error: [IF] No ENDIF
Test.asm(33): error: Unexpected: : nop : endif
Pass 3 complete
Errors: 32, warnings: 0, compiled: 61 lines, work time: 0.031 seconds

- - - Добавлено - - -

because pass1 and pass2 will see {$4000} as 0, only in pass3 there will be real value...

а сильно тяжело увеличить количество проходов?
мне кажется что 3 прохода это маловато для работы сложных генераторов кода

- - - Добавлено - - -

But if you want "array" of control variables,

мне нужен переменный набор переменных
чтоб сделать подобный код

vars = 3

a1 = 0
a2 = 0
a3 = 0

...
if a1 = 0
code
a1=a1+5
endif
...
if a3 = 0
code
a3=a3+10
endif

a1=a1-1
a2=a2-1
a3=a3-1

vars = 200

a1 = 0
...
a200 = 0

if a1 = 0
code
a1=a1+10
endif
....
if a200 = 0
code
a200=a200+44
endif

a1=a1-1
a2=a2-1
...
a200=a200-1

а дальше будет генерироваться адреса для org-ов

[Ped7g]

а сильно тяжело увеличить количество проходов?
мне кажется что 3 прохода это маловато для работы сложных генераторов кода

It's long term goal to make it N-pass assembler (I think we are pretty close, maybe even v1.14.0 or v1.15.0).
But `{ address }` will basically always return real value only in the last pass, again it's device memory, not script language.

I mean, there is already LUA, so as long as you will write the whole code generator in it, you are golden, even in current sjasmplus version, but you must do all the dynamic logic in the Lua, to emit only final stream of instructions without conditionals/dups/macros/etc.. = "blocks".

Of course it would be better if things like IF+ENDIF would work also in `_pc`, but it's problematic.

Basically I need to collect all `_pc()` calls into buffer, and **then** parse it similarly like "include", so if you have 100 lines of Lua script doing _pc(...), you would learn about 14th line using invalid instruction after the `endlua`, not at 14th `_pc(...)`, which is not good error reporting...

Or I need to rewrite whole internals to regular C++, where Lua script would be just another input stream like reading the file and the rest of assembler would have no idea the code is produced by Lua - I'm afraid this one will not happen due to my time (but maybe the sjasmplus/sjasmplus branch can make this one work, that source is now lot more proper C++ and better style quality, maybe there it needs only few changes)

Or maybe I can do another hack... add `_pc_buffered(...)` so people like you have at least some option, even if the error reporting will be delayed down to `endlua` line.

... still not much time, so I didn't study your ending example yet, sorry, will try to take a look during weekend or so.

- - - Updated - - -

I did take a quick look at the ending example, and I don't get it, can't tell which parts are unique and which can be calculated/generated... like why a1+5 in case of 3 vars and a1+10 in case of 200, and why do you even change those variables, if you don't repeat the whole block, etc...

Mind you, you can create dynamic amount 1..N of labels through a bit of hackery with DUP (with MAX_N repeats) and macros with partial name substitution, but those constant coefficients in those IF/ENDIF then has to be calculated or form some fixed array. But I guess the end result would be very difficult to read and maintain.

I'm still not sure why you don't use full Lua generator, if you need such complex stuff, I would expect it to be even easier to write, if you are experienced in Lua (unfortunately I am not, I'm just beginner).

- - - Updated - - -

And colons in _pc("nop : nop") are like different topic, that thing is maybe easiest to fix from this whole discussion ... (but still not that easy, even that may be complicated, probably not 2-3 lines fix).

[NEO SPECTRUMAN]

I'm still not sure why you don't use full Lua generator,

а как мне организовать переменный набор переменных в lua?
опять же было бы там _write_to_memory(16384.255)
и я бы как то выкрутился
да и документации на lua которая встроена в sjasm нет
неизвестно какие команды в ней есть а какие нет

может там как то можно завести массив на 256 значений
и использовать его

- - - Добавлено - - -

вот нагуглил такое

    a = {}    -- new array
    for i=1, 1000 do
      a[i] = 0
    end

пойду пробовать
заработает ли подобное в lua sjasm-a

- - - Добавлено - - -

	a = {}
	for i=1,1000,1 do
	a[i] = i*2
	end
	print (a[3])
	print (a[6])
	print (a[10])

на вид работает

SjASMPlus Z80 Cross-Assembler v1.13.2 (https://github.com/z00m128/sjasmplus)
6
12
20
Pass 1 complete (0 errors)
6
12
20
Pass 2 complete (0 errors)
6
12
20
Pass 3 complete
Errors: 0, warnings: 0, compiled: 18 lines, work time: 0.000 seconds

- - - Добавлено - - -

I'm still not sure why you don't use full Lua generator, if you need such complex stuff, I would expect it to be even easier to write,

ну это должна быть надстройка уже над готовым кодом
в которым тоже куча своих if endif
а результат работы lua кода должен быть специальный адрес для org
так что переписать на lua может не получиться...

- - - Добавлено - - -

I'm still not sure why you don't use full Lua generator

и как мне сделать на lua такое?


lua (use code_size)

code
defb $01
defb $01
code_end
code_size = code_end - code


lua (use code_size)

code
defb $02
defb $02
defb $02
code_end
code_size = code_end - code

lua (use code_size)

code
defb $03
defb $03
code_end
code_size = code_end - code

- - - Добавлено - - -

Ped7g, а еще в целях отладки не хватает pause
чтоб через display и print ()
можно было пошагово проследить
правильно ли идет сложная генерация кода

[Shiny]

а как мне организовать переменный набор переменных в lua?

ман почитать? нагородил тут не пойми зачем.

Условная компиляцыя:

 DEFINE rel

 IFNDEF rel
 include "../utils.inc"
 ELSE
 include "../demovars.inc"
 ENDIF

    display /d,end-begin
    display notecnt
 IFNDEF rel
	savesna "pair.sna",begin
 ELSE
	savebin "01pair.code",begin,end-begin
 ENDIF

как-то так

- - - Добавлено - - -

Он же забанен, вроде.

не, просто обидели мышку, накакали в норку - сверху темно, тесно и воняет.

[Ped7g]

...
и как мне сделать на lua такое?


lua (use code_size)

code
defb $01
defb $01
code_end
code_size = code_end - code


lua (use code_size)

code
defb $02
defb $02
defb $02
code_end
code_size = code_end - code

lua (use code_size)

code
defb $03
defb $03
code_end
code_size = code_end - code

https://github.com/z00m128/sjasmplus...s/lua_examples

        ;-----------------------------------------------------------------
        ; idea is from NEO SPECTRUMAN, who was trying to speed up "opcode" jumptable.
        ; implementation of Lua scripts and macros for sjasmplus is from Ped7g
            device zxspectrum48

        ;-----------------------------------------------------------------
        ; example of usage of the produced table (code provided by NEO SPECTRUMAN)
            org     $C000
        ; A = operation (alias "opcode") number 0..255
            ld      l,a                 ;4
            ld      h,high opJpTab      ;7
            ld      h,(hl)              ;7
            jp      (hl)                ;4
                                        ;=22t

        ;-----------------------------------------------------------------
        ; define LUA functions for memory allocations for opcodes functions
        ;
        ; (the ";" ahead of "end" and some "--" is not needed for Lua, but for my text
        ; editor sjasmplus syntax highlight, as it gets confused by lua source)
        ;
        ; Opcodes *must* be allocated in sequence (0,1,2 ...) to avoid large empty
        ; areas in memory, or even running out of memory completely. Also opcode
        ; implementation subroutines must be reasonably short (few bytes, not hundreds)

        lua pass1
            function allocateOpMemory(opcode)
                -- search for free "page" (512B pages starting at opRoutines address)
                freePage = _c("high opRoutines")
                while allocatedPages[freePage] and opcode < allocatedPages[freePage] do
                    freePage = freePage + 2
                    -- +2 to operate over 512 bytes, with 256B pages high opcodes like FE
                    -- may overwrite following page where early opcodes like 01 resides
                ;end
                ; -- remember it for "finishOpAllocate" function
                _G.lastFreePage = freePage
                ; -- free page found, emit it into jump table
                _pc(string.format("org $%04x", _c("opJpTab") + opcode))
                _pc(string.format("db $%02x", freePage))
                ; -- and reset ORG to target memory for opcode function body
                _pc(string.format("org $%04x", freePage*256 + opcode))
                _pl(string.format("opcode_%02x_impl:", opcode))
            ;end    -- ";" to make my Kate editor syntax highlight survive "end" in lua

            function finishOpAllocate()
                assert(_G.lastFreePage, "allocateOpMemory not called yet")
                allocatedPages[_G.lastFreePage] = _c("low $")
            ;end

            function setOrgAfterLastAllocated()
                checkPage = _c("high opRoutines")
                while allocatedPages[checkPage] do
                    lastAdr = checkPage*256 + allocatedPages[checkPage]
                    checkPage = checkPage + 2
                ;end
                assert(lastAdr, "no memory was allocated yet")
                _pc(string.format("org $%04x", lastAdr))
            ;end
        endlua

        ;-----------------------------------------------------------------
        ; helper macros to make the lua calls one-liners in asm
        macro allocateOpMemory _opcode?
@__allocateOpMemory_opcode = _opcode?
            lua allpass
                allocateOpMemory(_c("__allocateOpMemory_opcode"))
            endlua
        endm
        macro finishOpAllocate
            lua allpass
                finishOpAllocate()
            endlua
        endm

        ;-----------------------------------------------------------------
        ; global definitions and variables used to generate jump table
        
        ; jump table with "high" bytes of opcode function addresses
opJpTab     equ     $7F00               ; must be 256B aligned, size 256B
        ; opcode functions will go into memory starting from $8000
opRoutines  equ     $8000               ; must be 256B aligned, size dynamic (N * 512)
        ; reset all Lua global variables ahead of each assembling pass
        lua allpass
            allocatedPages = {}     -- reset allocated pages for every pass
            lastFreePage = nil
        endlua

        ;-----------------------------------------------------------------
        ; define opcode functions (builds also jump table and labels like "opcode_a1_impl")

            allocateOpMemory 0
            db      1, 2            ; fake "implementation" (just 1,2,3,4,... byte values)
            finishOpAllocate

            allocateOpMemory 1
            db      3, 4, 5
            finishOpAllocate

            allocateOpMemory 2
            db      6, 7, 8
            finishOpAllocate

            allocateOpMemory 3
            db      9, 10
            finishOpAllocate

            allocateOpMemory 4
            db      11
            finishOpAllocate

            allocateOpMemory 255
            db      12, 13, 14, 15, "this goes over into page $8100..81FF"
            finishOpAllocate

            lua allpass
                setOrgAfterLastAllocated()
            endlua

        ;-----------------------------------------------------------------
        ; store result as binary blob for simple verification in hexa editor
            align   512, 0              ; fill also last page to full 512B first
            savebin "lua_build_jp_table.bin", opJpTab, $ - opJpTab

I think this should work reasonably well... I had to raise "page" size to 512B to avoid memory overwrites from last opcodes like FF (with 256B you have either only 1B in "current page", or you need to know that following "page" was used for bigger opcode and beginning of the area is unused. So the memory efficiency will be not very good, probably having multiple almost-256B free blocks between, but on the other hand you don't need to place them manually... So as long as you can exchange a bit of memory for a convenience, this should be fine.

[NEO SPECTRUMAN]

allocatedPages[_G.lastFreePage] = _c("low $")

С low $ не будет правильно работать!
нужно так

op_allocated_pages[_G.op_allocator_last_free_page] = (_c("$")-(_G.op_allocator_last_free_page*256))

(я у себя изменил названия всех переменных под стиль своего исходника)

while allocatedPages[freePage] and opcode < allocatedPages[freePage] do freePage = freePage + 2

это сильно не экономично!

lua_build_jp_table.asm

лучше переименовать в lua_build_fast_jp_table.asm
чтоб было очевидно что там не обычный вариант


и все таки нужно как то измерять количество байт в процедуре
между allocateOpMemory и finishOpAllocate

завести массив с минимальным начальным адресом в странице

чтобы можно было предотвратить перезапись кода
и переместить процедуру в следующую свободную страницу



может будет проще
добавить средства генерации таких таблиц в сам sjasm?

например мне нужно будет делать несколько таких таблиц одновременно
при этом остается много места
куда можно было бы поместить другие процедуры
lua здесь уже не поможет
или нужно индексировать каждый байт
и искать свободное место немного по другому

[Shiny]

...там еще некоторое обсуждение велось в личке
так что вникай сильней

буду краток: не могу, у меня лапки.

за одно скажи видел ли ты где то такие таблицы переходов?
(именно ld h,(hl) а не кастрированный ld l,(hl) который иногда применяется)
(я не видел))

не помню. самое извратское, что я применял - стек для чанок.

например мне нужно будет делать несколько таких таблиц одновременно
при этом остается много места
куда можно было бы поместить другие процедуры
lua здесь уже не поможет

а написать процедуру религия не позволяет?

[Ped7g]

С low $ не будет правильно работать!

Indeed, good catch. I will probably fix it by using _c("$&0x1F") in the official example, but your fix is correct too. (didn't verify my idea yet, but should be ok)

это сильно не экономично!

LUL. "... compiled: 119 lines, work time: 0.002 seconds" ... it's even less economic to comment on this point...

и все таки нужно как то измерять количество байт в процедуре
между allocateOpMemory и finishOpAllocate

I'm not sure if I understand you... while inside the opcode-function code, you can measure byte usage at any particular spot by `code_size = $ - opcode_XX_impl` where "XX" is hexa opcode in lowercase (i.e. "opcode_1a_impl" for opcode 26 == 0x1a)

I would need code size ahead before allocating memory for more optimal solutions, and that *is* problem. It can be probably worked around by defining each subroutine first as macro, and then assemble all of them twice, first time at $0000 to collect code size, second time at final destination. But such solution requires lot of cooperation, the macros for subroutines must be written in certain way, etc... not a prefect fit for generic example.

завести массив с минимальным начальным адресом в странице

чтобы можно было предотвратить перезапись кода
и переместить процедуру в следующую свободную страницу

*fresh new idea* Thinking about it... I can actually collect code-sizes during pass1 (unless the subroutine does something stupid with if/ifdef/dup to modify it's size between pass1 and pass3)... and use that info in pass2 and pass3 to create the final placement.

That should be possible with current sjasmplus, maybe I will try to create it as "version 2" somewhere in the future.

может будет проще
добавить средства генерации таких таблиц в сам sjasm?

I don't see how to fit that into sjasmplus in some nice versatile generic way. Also I already forgot all the "algorithm" stuff from the high school, so I can't even recall what was the optimal solution to "pack the boxes"... (can't even count how many times we had similar task at programming contests, hehe)

If somebody will show me some good designs, how to do it (I mean designs of everything: syntax, docs, algorithm), and it will be compatible with current implementation, then it may happen, but so far only very few people did any kind of "pull request" with new improvements, and people who want to try must be prepared that I will comment on their code a lot and it may take several iterations before the code will be accepted.

At this stage of development I don't want to spend my time on this, it's very specialized, while I have major general issues with labels, big memory and define substitutions.

например мне нужно будет делать несколько таких таблиц одновременно
при этом остается много места
куда можно было бы поместить другие процедуры
lua здесь уже не поможет
или нужно индексировать каждый байт
и искать свободное место немного по другому

Lua is full scripting programming language. If your requirements needs O(N*M*K*J) complex algorithm, then so be it. Who cares. We are talking about hundreds of subroutines. I.e. even some ugly brute force O(N^4) on modern CPU will take maybe 10s to generate final machine code. I don't care. In my paid work right now I have Android Studio and I'm forced to use corporate notebook with windows and antivirus, so only "clean" of the project needs 20+ seconds (on my home notebook with linux the "clean" of large projects takes usually between 0.5s to 3s and most of that is that f*cking gradle starting up), and full rebuild takes 4 to 7 minutes. If I would have project build times at 10s, I would be super happy.

The point is, that you *can* do it in Lua. You don't even need sjasmplus at all, you can just write machine code into file straight from the Lua script. It will be probably ugly and difficult to debug and maintain (the generator in Lua), and I would probably rather write it in C++, but stop crying in the way "it's not possible to do it in lua". It's just difficult and ugly, but by the Turing computational theory, if you can clearly (in computational way) specify the desired outcome, it surely can be calculated also in Lua. Or in Z80 machine code. Or in PHP. Or in hundreds of other programming languages.

The question is not if it is possible, but which tool is best fit. For such complex generators the sjasmplus is currently a bit cumbersome. If you can do it better in some other assembler, just use it. Programming languages, compilators and assemblers are just tools. Pick the best one for your current task.

And if you need more runtime memory, use ZX Spectrum Next ... ... 756kiB in base model, and some people will have them extended to 1.75MiB.... :P (or you can code for x86_64 and have gigabytes of memory available).