Jump to content

  1. Current Location:
  3. »
  4. »
  5. »
PureZC Community Rules and Guidelines
Photo

Some System Limit Questions...

Started by Timelord , Dec 07 2015 11:06 AM
wiki values limits maximums scripting zscript zquest zasm lexer stack

  • Please log in to reply
29 replies to this topic

#16 Saffith

Saffith

    IPv7 user

  • Members

Posted 11 December 2015 - 03:21 PM

There seems to be some miscommunication here.
 
for (int i = 0; i < 30; i++)
{
	int MehX = i / 4;
	int MehY = i / 2;
	Screen->DrawCombo(3, Ghost_X + (MehX / 2), Ghost_Y - MehY, DISCO_COMBO + Ghost_Dir, 1, 1, Ghost_CSet, 16 - MehX, 16 + MehY, 0, 0, 0, -1, 0, true, OP_TRANS);
	Ghost_Waitframe(this, ghost); //We're doing drawing and movement set-up here; I believe this would work.
}
for (int i = 10; i > 0; i--)
{
	int MehX = (i * 3) / 4;
	int MehY = (i * 3) / 2;
	Screen->DrawCombo(3, Ghost_X + (MehX / 2), Ghost_Y - MehY, DISCO_COMBO + Ghost_Dir, 1, 1, Ghost_CSet, 16 - MehX, 16 + MehY, 0, 0, 0, -1, 0, true, OP_TRANS);
	Ghost_Waitframe(this, ghost);
}
for (int i = 0; i < 60; i++)
{
	int MehX = i * 4;
	int MehY = i / 4;
	Screen->DrawCombo(3, Ghost_X - (MehX / 2), Ghost_Y + MehY, DISCO_COMBO + Ghost_Dir, 1, 1, Ghost_CSet, 16 + MehX, 16 - MehY, 0, 0, 0, -1, 0, true, OP_TRANS);
	Ghost_Waitframe(this, ghost); //We're doing drawing and movement set-up here; I believe this would work.
}
There are nine variable declarations there - three each of i, MehX, and MehY - so they use nine places in the stack.

#17 grayswandir

grayswandir

    semi-genius

  • Members

Posted 11 December 2015 - 04:21 PM

What about i? Is that taking a space up?


#18 Saffith

Saffith

    IPv7 user

  • Members

Posted 11 December 2015 - 04:28 PM

D'oh, yeah, overlooked those. Edited.

#19 Timelord

Timelord

    The Timelord

  • Banned
  • Location:Prydon Academy

Posted 11 December 2015 - 05:04 PM

There seems to be some miscommunication here.
 

for (int i = 0; i < 30; i++)
{
	int MehX = i / 4;
	int MehY = i / 2;
	Screen->DrawCombo(3, Ghost_X + (MehX / 2), Ghost_Y - MehY, DISCO_COMBO + Ghost_Dir, 1, 1, Ghost_CSet, 16 - MehX, 16 + MehY, 0, 0, 0, -1, 0, true, OP_TRANS);
	Ghost_Waitframe(this, ghost); //We're doing drawing and movement set-up here; I believe this would work.
}
for (int i = 10; i > 0; i--)
{
	int MehX = (i * 3) / 4;
	int MehY = (i * 3) / 2;
	Screen->DrawCombo(3, Ghost_X + (MehX / 2), Ghost_Y - MehY, DISCO_COMBO + Ghost_Dir, 1, 1, Ghost_CSet, 16 - MehX, 16 + MehY, 0, 0, 0, -1, 0, true, OP_TRANS);
	Ghost_Waitframe(this, ghost);
}
for (int i = 0; i < 60; i++)
{
	int MehX = i * 4;
	int MehY = i / 4;
	Screen->DrawCombo(3, Ghost_X - (MehX / 2), Ghost_Y + MehY, DISCO_COMBO + Ghost_Dir, 1, 1, Ghost_CSet, 16 + MehX, 16 - MehY, 0, 0, 0, -1, 0, true, OP_TRANS);
	Ghost_Waitframe(this, ghost); //We're doing drawing and movement set-up here; I believe this would work.
}
There are nine variable declarations there - three each of i, MehX, and MehY - so they use nine places in the stack.

 

 

Right, nine registers. That shouldn't be enough to cause an overflow (they do overflow); or is there a separate limit on stack space, other than the gd regs? It acts as if it's using 203 gd registers here.


Edited by ZoriaRPG, 11 December 2015 - 05:07 PM.

#20 Saffith

Saffith

    IPv7 user

  • Members

Posted 11 December 2015 - 05:42 PM

There are nine in that excerpt. The full script had many more.

#21 Timelord

Timelord

    The Timelord

  • Banned
  • Location:Prydon Academy

Posted 12 December 2015 - 01:10 AM

There are nine in that excerpt. The full script had many more.

 
Right. The thing is, that if you compile it, and run it, it overflows, so they aren't popping off as expected. :/
 
It looks like you found that out though... Here's the tricky thing: Moving the int Meh* declarations out of the for loops, and directly into the run function freed up enough space to prevent it from overflowing. I'll need to check, but I doubt it was overflowing by only a few registers there, but I haven't looked at the ZASM output of either code file.

 

Ultimately, placing loops like this in their own functions, and then calling them from a script would provide better management of available registers. That at least, seems to be true.

 

If only there was a way to view live register usage during ZASM execution... That would be somewhat helpful.


#22 Saffith

Saffith

    IPv7 user

  • Members

Posted 12 December 2015 - 01:54 AM

There were over 200 variables declared in the original script. The functions it calls weren't included, but it's not hard to imagine one of them had enough to push it over. The fixed version has about 160 variables. Still a lot, but it's not a small difference.

#23 Timelord

Timelord

    The Timelord

  • Banned
  • Location:Prydon Academy

Posted 12 December 2015 - 02:27 AM

There were over 200 variables declared in the original script. The functions it calls weren't included, but it's not hard to imagine one of them had enough to push it over. The fixed version has about 160 variables. Still a lot, but it's not a small difference.

 

Hell, you're right. A cursory count put it at 201, plus function calls. It's the way it was constructed, purely. All those for loops each using one variable in the scope of run() though, doesn't help. So, what I get out of this, is that a variable declared in a statement within a function doesn't fall out of scope until the function exits, or it's replaced in the scope (statement) in which it was declared. That's silly, but I can live with it.

 

I try to bind my vars in arrays, and when I can't do that, I just use different function call parameters to utilise alternate functions in which I store the different sets of vars (usually strings). I think what got me, is that the compiler itself didn't spit out an error regarding register use in that script. I know that it can do that, but it obviously doesn't calculate that either for vars not at a global scope, or it doesn't take function calls into consideration, when determining if the script will exceed the available registers.

 

It didn't help that the script was overflowing as soon as it hit the first loop, either. I knew that, primarily because of the TraceS() details that I added, to debug it for Dimentio. It reached Phase 1, and then overflowed. :shrug:  I'm not sure why it made it precisely that far, but it certainly threw me off course.


#24 Saffith

Saffith

    IPv7 user

  • Members

Posted 12 December 2015 - 02:33 PM

The compiler can't detect stack overflaws. Try this, if you like; it will compile just fine:


Global and local variables are fundamentally different in ZScript. You get an error when you use too many global variables because gd256 isn't a valid register. That's not the case with local variables; PUSHR is valid no matter how many times it's repeated.

#25 Timelord

Timelord

    The Timelord

  • Banned
  • Location:Prydon Academy

Posted 12 December 2015 - 03:53 PM

The compiler can't detect stack overflaws. Try this, if you like; it will compile just fine:



Global and local variables are fundamentally different in ZScript. You get an error when you use too many global variables because gd256 isn't a valid register. That's not the case with local variables; PUSHR is valid no matter how many times it's repeated.

 

 

That's what I figured. Once I can start on the updated sources, I'll go over the compiler/lexer again. I might have some volunteers now, and we may be able to improve this sort of thing, ass some intelligent error finding, and warning, and do other improvements; although the simplest answer is to expand the bytecode for 2.55.

 

Did you note that problem with MIN_CONSTANT over on AGN? That's a bug I've previously not noticed. It's hardcoded (somewhere) to -021473.0001, not -214747.9999. Somebody slipped on that, likely very long ago.

 

I've also given a tiny bit of thought to 2D arrays. They should be possible by stacking Get/SetGlobal/ScriptRAM. If I can document how that works, it may even be possible to do them manually.


#26 Timelord

Timelord

    The Timelord

  • Banned
  • Location:Prydon Academy

Posted 17 December 2015 - 06:28 PM

2^32, at least effectively. I don't know if there's actually a limit on the number of instructions a script can compile to, but the program counter is an unsigned int, so any instructions beyond 2^32-1 can't possibly run.
 
Only limited by the number of instructions, as far as I know.

 
I think in this case, it's a true int, 2^16; not 2^32 (i.e. not a fix).
 
'Invalid ZASM command 65535 reached' ...

.................is reported if you try to do something like this:
 

int x;
for (int q = 0; q < 200000; q++ ) x = Rand(4096);

I guess I now know the limit, and it's somewhat frightening (small) for some unorthodox tasks. As an example, it means that if i declare an array with a size of 100000, that I cannot parse it in a for loop with SizeOfArray(), or at all really, in one loop, so it's actually a problem, albeit a minor one; and I'll amend the entry.

 

 

I can't imagine changing the programme counter over to an unsigned fix would affect it in any negative way, so that's something for me to look into for 2.55  / 2.60 / 2.99999995. Actually, I'm not sure, but I wouldn't expect that to break anything if it was done with the present quest and save formats, either, as that is purely on the ZC 'player/viewer' side of things. .


Edited by ZoriaRPG, 17 December 2015 - 06:41 PM.

#27 Saffith

Saffith

    IPv7 user

  • Members

Posted 17 December 2015 - 07:27 PM

That's an invalid instruction, not too many instructions. And that's almost definitely a bug. If you'll show me the full script, I'll look into it, though I doubt it's something I can fix.

A loop with more iterations doesn't compile to more instructions. It runs more instructions, but that's different. There's no limit on that.

I know for certain that the limit on the total number of instructions greater than 65535. That used to be the limit, but people were actually able to reach it (LtM and Nick did, at least), so it was increased. I am just as certain that you can run hundreds of thousands at a time, because we count them as part of checking for F9/F10 while hung.

#28 Timelord

Timelord

    The Timelord

  • Banned
  • Location:Prydon Academy

Posted 18 December 2015 - 04:06 AM

You know, I didn't save it, so, I reopened ZC and ZQ, retyped the same premise, and it didn't give me the error. Perhaps it was some kind of memory corruption from running ZC and ZQ on Win7 over a long period...

 

I did however, find a way to force that error again, by suspecting the only possible error type that I could have had there. and this one reporting the limit reached, but for a very good reason:

 

for ( int q = 0; 2 < 214747; q++ )

 

That type of error never reported this in the past, so it might be new, and perhaps I had a typo like that in the original code, but I doubt that somehow. I'm going to see if there are any .qb files with it... Rats, the timed auto-saves are all too new. I usually don't overwrite, but as the entire point was to quickly stress-test some values, it was the fastest way to do it.

 

I did just re-do it, and verify that 214747 works, so I'm at a loss. I know that I error-checked the code as soon as I saw it, but perhaps I wasn't looking for the right thing. In any event, it is 2^32-1, an infinite for loop nor reports that error, and you can escape it with F6. That's certainly new, and very nice indeed.

 

I'm, going to do some time-based tests on how long a single loop iteration of that size takes, if reading from, or writing to specific indices from an array that size has any impact on r/w/s time, and how long it takes to write x lines to the log. I'm not doing it for fun, as people do ask me these questions, and not merely one person, on one occasion.

 

I don;t believe that array size should impact reading, or writing values to specific indices, but the loop iteration seems a bit overlong. I presume this is because of how ZASM has to retrieve and store the values. I also need to compare time tests for reading from, and writing to all screen combos, and such, as I've been told by more than one person that this is slow, but AFAIK it isn't. it should in fact be faster than doing the same with a ZScript array, unless it isn't when the ZScript array pointer is valid.

 

Still, it should not be slow.

 

Some notes on performance, and stress tests that I've done...for those interested:

  • A single task loop of randomising 214747 values took approximately 2.8 seconds, if anyone cares.
  • Reading 214747 values from an array and storing them to a temp val in a for loop took ~1.4 seconds, or less, during which on my Core i5, running framecount dropped to 48.
  • Writing a value of 10 to 214747 array indices took 2 seconds, and framecount dropped to 25.
  • Reading 214747 array indices and copying each to the same index position in a second array also took 2 seconds, and framecount dropped to 9fps.
  • Writing to each position of ComboD, ComboC, ComboS, ComboT, ComboF, and ComboI, six times over each, took all of about 0.25s.
  • Writing the values of each to an array, took about the same, less than one second to read 6336 Screen->Combo*[] values, and write them into an array with nested loops.
  • Drawing 999 Tiles, or Combos, with randomised selection from a large sample size, and randomised positions, randomised rotations, and other random factors is instantaneous, and allows ZC to run at full speed (throttled).
  • Drawing 1000 pixels at random locations (any pixel on the screen, x or y), out of a list of 24 random colours, ran at full speed (throttled). It also looked somewhat like a kaleidoscope.
  • Drawing 1000 lines, with every value except layer, opacity, and scale, randomised, ran at 60fps. (This called eight Rand()s.per instruction.)

 

 

 

I will need to re-do these test again, and some othersnin the future, as there is a reasonable bias in how many other tasks I have running, other then ZC, so my system is fairly bogged down at present. These are only preliminary values, and I'll see about doing a proper benchmark sometime later this month. I'm going to need to do it on at least two sets of HW, and more than one OS.


Edited by ZoriaRPG, 18 December 2015 - 05:16 AM.

#29 Lejes

Lejes

    Seeker of Runes

  • Members
  • Location:Flying High Above Monsteropolis

Posted 18 December 2015 - 08:45 AM

I would recommend looking at uncap speeds. ZC isn't hugely CPU intensive to begin with, and only checking to see if it maintains 60 FPS can let minor performance hits go unnoticed. Drawing 999 random tiles every frame cut about 210 frames per second from my uncap speed, for example. It's not hugely relevant if you only have one such script, but things like this can combine and slow down a quest, especially if someone has an older machine.

#30 Timelord

Timelord

    The Timelord

  • Banned
  • Location:Prydon Academy

Posted 18 December 2015 - 08:58 AM

I would recommend looking at uncap speeds. ZC isn't hugely CPU intensive to begin with, and only checking to see if it maintains 60 FPS can let minor performance hits go unnoticed. Drawing 999 random tiles every frame cut about 210 frames per second from my uncap speed, for example. It's not hugely relevant if you only have one such script, but things like this can combine and slow down a quest, especially if someone has an older machine.

 

The main problem there, is that an un-throttled speed is largely extremely variable. I prefer to check a throttled speed on varying degrees of hardware, downscaling the system, until it starts to reach a handicap. I'm already testing this on the sort of hardware that most members here would find laughable, too.

 

You are of course right about stacking effects, and while I do plan to test for cascading loops, and instruction series at some point, it's extremely unlikely that most people will reach one of these caps, and in the case of drawing, you can't exceed 1000 draws, so testing how 1,000 draws of various types perform, when they rely on random functions, is useful in making a determination as to whether any specific type is more demanding than another type.

 

Checking actual throttled frameloss when running an instruction that operates in one frame, is also essentially relative. It will use x-frames of clock time to do what it needs to do. if it does it throttled, it will also do it un-throttled.

 

Un-throttled checks are also even more varied by how much idle time is available on the CPU, and bus side, whereas throttled speed is usually reserved by the programme, to an extent. I do note if the un-throttled speed takes a hit, but it's something that--on this system--will vary by the minute, hour, or day so wildly that it has proven entirely unreliable as a benchmark test. I would need to do hourly reboots, to perform that kind of extensive benchmarking, and my main goal is to verify if under normal operation, any series of instructions will pose a greater risk in use, than another.

 

Bear in mind that I do repeat the throttled tests at different times, and compare, then average them. Without a process/thread inspection tool, that can read what Allegro is doing, it's a pretty tedious affair too.

 

If you want to participate in a multi-user benchmarking determination, I'd be happy to discuss that. Otherwise, i will probably set up what I feel to be reasonably deep, nested chains of instructions, functions, and declarations, and try those under different stress conditions. Something that reflects some of the more complex engine routines that I've made, and that others have made, that is usually the cause for concern.

 

214714 instructions per frame, though, drives a stake through ZC's heart on anything that I own. I;d be curious what someone with a 'serious gaming system' would report under identical conditions, but I'ven't one of those, and the next best thing is to run it on some servers, and see how they handle it, as these routines are primarily mathematical in nature.

 

I also highly suspect that the interface for Allegro 4 compatibility on Allegro 5 (Linux, OSX) will be a huge process-time sink for all of this..

 

In any event, demonstrating that ZC can reach these high thresholds and not fail to run at normal speed, on old HW, is mainly a way of verifying that most of the user complaints that ZC slows down when they issue 50 to 100 draws of some kind, or use large arrays, or redraw large screen areas, is more likely due to user (script) errors, than anything internal. If I encounter something that does seem to highly impact ZC, then I know it's something to flag.

 

At the least, it may help questmakers, and scripters to determine minimum system capabilities for advanced quests.


Edited by ZoriaRPG, 18 December 2015 - 09:17 AM.

Back to Scripting Discussion

Also tagged with one or more of these keywords: wiki, values, limits, maximums, scripting, zscript, zquest, zasm, lexer, stack

1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users

  1.  
  2. Privacy Policy
  3. PureZC Community Rules and Guidelines ·