Disclaimer: This documents is not intended to be some kind of sacred text you should follow. It is just an explanation of how I code with some stuff I've thought that could be useful for you to take in account. Use it at your own risk. Feel free to modify any mistakes you find.
PUSHING QB TO ITS MEMORY LIMITS
A TUTORIAL ABOUT MODULAR PROGRAMMING, GLOBAL VARIABLES AND MANUALLY COMPILING AND LINKING *
Well... When I decided to try the QB 4.5 compiler a year and a half ago (I was using QBasic 1.1 for a while, and then C) I found myself inmersed in an unknown sea of unwatchable variables, uncompilable code, weird memory screws and many things like that.
There are no tutorials which explain how to solve these issues clearly (at least no tutorials which I know). I had to figure out every technique by myself. And to fill that void relating to modular programming and global variables I've decided to write this short tutorial.
English is not my first language and you'll probably notice it very soon, but I'm doing it as good as I can. If there are incorrections, feel free to fix them and please email me the correct version of this tut. Thank you in advance.
#1 Let's go to it: Modular programming?
Modular programming basis is to split the program code into several parts, called `modules'. This programming philosophy started a whole lot of years ago, when computers were not so good as they are nowadays. When a programmer wanted to work in a bigger project, he was forced to split it in parts for the following main reasons:
* Memory issues: Older computers had to store both source code and object compiled/assembled code in main memory (RAM) and the amount of memory
was not very big.
* Time issues: Splitting in parts makes the compilation faster, as you only have to re-compile the code module you've changed, not the whole code.
Modular programming needs planning. Spaghetti code tend to make extensive use of global variables and global variables are a problems in therms of modular programming.
We all know that when you define a SHARED variable with DIM SHARED it can be read and written from within every FUNCTION or SUB. But if you are using several modules you will find that these global variables are not so global 'cause they can be read and written *only* from within FUNCTIONs and SUBs from *the same module*. It looks like a problem for many programmers as I have been reported. It isn't a real problem 'cause there are many object oriented techniques which can be applied to non-object oriented programming (to QB) which solve that issues. In addition, real global variables *can* be done. I'll discuss how to do it in a later section.
A program splits in module contains one MAIN module with the program's entry point, every other module only contain SUBs, FUNCTIONs, SHARED variable definitions and DATAs.
Each module has a .BAS file with the code and a .BI file with the DECLARE SUBs and DECLARE FUNCTIONs. If you want to use those FUNCTIONs or SUBs from within another module, you need to '$INCLUDE' the .BI file in order to make that code visible.
When you start a QB program, you first code the MAIN module, and then add the other files. This is called 'making a project'. In the FILE menu you will find 'Create file' to start coding a new module or 'Load File' to load one existing. Once you save and exit QB, it creates a .MAK file which tells what modules to be included. This file names after the MAIN module's filename and it is loaded automatically when you load that module.
It's useful to group the files into 'thematic' modules, that is, a module for GFX, a module for SOUND, a module for the game ENGINE... This makes possible to share the project among many people. For example, you're coding the main ENGINE and you only need to know what functions to call for drawing the GFX. The GFX module can be coded by other person and you don't need to know how it is coded, only the .BI file. It implies that you can code the main ENGINE which uses GFX without having a GFX file coded, only with the .BI file (it won't run, but at least you can make your work without having to wait the other coders to do theirs).
#2 Observer FUNCTIONs and modifier SUBs.
Imagine that you have a module to control the player's statistics in a RPG game. That module have global SHARED variables to store all the player's values, and also have the battle engine, for example.
It's clear that you need to modify and read these values from the main engine, but the engine is in another module and here comes the problems... these SHARED variables are not visible from the main module. What can be done? Well, let's take a look to an Object-oriented programming book and we find that the modules scheme is very similar to an object-oriented scheme.
In object-oriented programming you deal with objects. An object contains variables to store data and functions to work with that data. It looks alot like our modular programming scheme, although it isn't the same thing (JAVA and C++ programmers know what I'm talking about). When a programmer needs to modify or read the variables (called attributes) from an object he or she cannot do it directly. He or she needs to use observer FUNCTIONs and modifier SUBs.
Look at this code [DUMMY1.BAS]:
'$INCLUDE: 'DUMMY1.BI' ' <- The DECLAREs
DIM SHARED dummyA% ' <- a shared variable
SUB foo ' This subs makes nothing useful
PRINT dummyA% ' It can be done, 'cause DummyA% is
' visible from here
' If we want to write a value to DummyA% from another module we have to
' use this SUB
SUB writeDummyA (value%)
dummyA% = value% ' This writes to DummyA%
' If we want to read DummyA%'s value from another module we have to use
' this FUNCTION
readDummyA% = dummyA%
We should write another file called DUMMY1.BI containing this code:
DECLARE SUB foo ()
DECLARE SUB writeDummyA (value%)
DECLARE FUNCTION readDummyA% ()
Now let's code a MAIN module, which will give us an entry point to the program. If we would like to change 'DummyA%' from DUMMY1.BAS we cannot write directly DummyA% = 7. It wouldn't work. To change DummyA% we need to call writeDummyA:
'$INCLUDE: 'DUMMY1.BI' ' <- This makes the code visible
writeDummyA 227 ' Write 227 to DummyA%.
PRINT readDummyA% ' This prints DummyA%'s value.
This doesn't seem very useful, but dealing with player's values in a RPG game it is. For example, we could have a readDexterity% FUNCTION and a modifyDexterity SUB.
The SUBs are called `modifiers' 'cause they are used to modify the module's own variables.
The FUNCTIONs are called `observers' 'cause they are used to 'observe' the module's own variables.
This way you can do global variables without to do 'real' inter-module globals...
#3 REAL INTER-MODULE GLOBALS
Yes, it can be done. You can define globals visible from within any SUB or FUNCTION from within any module. I don't like it 'cause it usually produces spaghetti code. Remember that the use of globals *always* can be avoided using observers and modifiers.
But oh, well, just for you to know. I'll explain how to do it. I haven't tried it as much as I would have liked to, but I think it works. You have to declare the variables in a .BI file and then use the COMMON SHARED to make them fully shared:
COMMON SHARED asmatic%, cantbreathe%
COMMON SHARED Player AS DType, Name AS STRING
And now you have to include this file in every module. But, oh gin, it has restrictions! Yes, a whole lot:
* QB doesn't allow to use more DECLAREs if you have written any 'executable' code. Those DIMs and the COMMON SHARED statement are 'executable code'. Whe QB finds an '$INCLUDE what it does is *literally* add the .BI code to the .BAS code. If you have executable code in a .BI file you won't be able to include any other .BI file after it, so the '$INCLUDE: 'GLOBALS.BI' in our example *has to be* the last $INCLUDE in every module.
* These global variables take LOTS of base memory as they are always there. I don't know how exaclty QB manages memory, but in C these globals are in the same DATA segment so it limits storage. It's better not to use ANY global variables.
#4 AVOIDING OUT OF MEMORY ERRORS
This is very likely to have to deal with DIMs, globals and misuse of dinamic variables.
A) Problems with DYNAMIC ARRAYS.
We have been told that putting '$DYNAMIC at the beginning of our code we have 'more memory' although it is not completely true. Let's explain the difference between '$DYNAMIC and '$STATIC.
Static arrays are 'created' in compile time. It means that when you have static arrays, the compiler makes room from them which implies faster access but less storage capacity.
Dynamic arrays are 'created' in run time. It means that when you have dynamic arrays, the compiled program reserves memory when it is running. It means slower access but more capacity 'cause you can reserve as many memory as you want whenever there is free memory enough.
But there is a problem... Imagine this:
FOR i%=0 TO 1000
SUB dunnoThis (a%)
FOR i%=0 TO 100
This program is completely useless but it would help you to see what the problem is. The dunnoThis SUB is called 1000 times. The first time it is called it creates an array of 1000 integers, this means that it reserves 2000 bytes of memory and points them with data%. The second time the SUB is called it finds DIM again so it reserves 2000 bytes again... and so forth. Probably after only a few iterations the program will crash with an 'Out Of Memory' error.
What can we do? easy. As long as dynamic memory is not needed anymore, you only have to free it. It is done with ERASE. Adding ERASE datas% at the end of dunnoThis code will solve the memory error.
I have had many out of memory errors for having forgotten a single ERASE, so if you have problems check your code.
B) Other ARRAY problems:
Place every DIM at the beginning of the code. When I was coding JILL it happened that the menu worked fine when I run the game. I selected a level, and if I returned back to the menu the game crashed. The problem is that I missed a DIM in the middle of the code and it was reserving memory again and again and again... Move every DIM to a safe place.
C) Avoid globals
I mean *every* globals. It's better to use dynamic variables. If you create a variable inside a SUB it is destroyed (and the memory it takes is freed) when you exit that sub. This means that in every time you are only taking the extrictly neccesary room.
Globals can be avoided adding parameters. For example, you have a global with a graphic and you have a SUB which prints it:
DIM SHARED graphic%(400)
You can avoid the global variable very easily, just pass it as a new parameter:
Now graphic%() doesn't need to be a global.
This technique has solved a lot of problems to me. Coming back to JILL, in case you've seen the Pong scene, I had a global array in the PONG.BAS module to draw the numbers. When the code grew it didn't worked at all. I deleted the global, defined the array from the PONG main SUB and then passed it as a parameter to the drawing functions and it worked again. So think about it.
D) Think before you code
And you won't have any problem and you won't have to move a comma with lots of memory errors.
E) Use MIDIs... BWSB sounds better but it takes lots of memory
F) If you use third party libraries (like DIRECTQB or FUTURE.LIB) build
them with only the SUBs and FUNCTIONs you're gonna use.
For example, I used DIRECTQB for JILL. When I built it I left out the blender functions, the 3D functions and the file managing functions, as well as the datafile support. It resulted on smaller .QLB and .LIB files and it always helps.
G) Also use a GFX lib instead of GET / PUT
When you're making sprites you need a Sprite array for the graphic and another one for the mask. In DIRECTQB or Future.LIB and other graphic libs you only need one array. This saves memory ;)
#5 OH NO! MY IDE GOT SCREWED
Yes, when your game becomes larger you won't be able to use your QB IDE. When I was coding Jill, I had to recode one level code 'cause the IDE didn't want to save the source to disk: it hasn't memory enough to do it... I got really angry. I had to code the last 30% of Jill from Dos EDIT.
Such inconveniences happen everyday when dealing with such a limited IDE. Although PDS has EMS sollutions I've been warned that they don't seem to work very well [thanx Claydragon]. The only working sollution is exit to DOS and run BC and LINK from the command line.
I don't have an extensive knowledge about BC and/or LINK. I only know what I have to know, 'cause I had to learn it when I was coding Jill.
Once you've edited your BAS files, you have to create OBJ files from them, just run BC for each one. I use the following options (the same used by the IDE):
BC filename.bas /O/T/C:512
It is more comfortable to create a BAT file. Open EDIT and write the following text, then save it as [RUNBC.BAT]:
Now, if you have for example three modules: GAME.BAS, GFX.BAS and FOO.BAS you have to compile every module:
Now you have the OBJs. To make a single EXE from them you have to LINK them together and add the QB runtime. This is done with LINK.EXE. LINK can have a very long commandline options when there are many modules, so it is better to use a script file. Create a file like this, and name it, for example, [GAME.LNK]:
*** NOTE: I know I need some corrections here ***
/EX /NOE /NOD:BRUN45.LIB GAME+
Wow!!! what's that?? It's weird. I'll explain what you should write:
* On the first line, place /EX /NOE /NOD:BRUN45.BI and the name of the main module, GAME in the example. If there are more modules, add a + symbol at the end.
* On the following lines, place the name of the other modules, adding a + symbol at the end of the line if there are more modules. Not that in third line, as FOO is the last module, we haven't written another + symbol.
* On the next line, write the name of the EXE file, GAME.EXE in our example
* Then leave a BLANK LINE
* Then list all the LIBs you're using. BCOM45.LIB comes with QB and it has to be included. If you use other libs, such as DIRECTQB, just add them on following lines, and don't forget to use the + symbol at the end of each line when there are more libs.
* Maybe you will have to include full paths for every file (the LIBs used).
Now you have your script file for LINK. To run LINK with it just...
In our example LINK @GAME.LNK
In theory, if you make good use of variables and free memory when you are not using it, you can make much longer programs compiling 'by hand'. I have reached tops of 250K EXE files which are impossible to manage from within the IDE.
#6 I NEED TWO QLBs... WHAT SHOULD I DO??
Yes! You have to mix them! I had to do it 'cause I wanted my QMIDI.QLB/LIB and DQB.QLB/LIB... So you have to use LIB.EXE and LINK.EXE to do it. Here is what I had to do...
A) First create a new .LIB file. We are using DIRECTQB as a base:
LIB MYLIB.LIB DQB.LIB;
Now we have MYLIB.BI with will be a copy of DQB.LIB.
B) Now we add MIDI.LIB:
LIB MYLIB.LIB + MIDI.LIB;
C) If we want to use more libs, just repeat B) with all these libs. This B) step can be done with .OBJ files as well. If you are not using DIRECTQB or FUTURE.LIB you will want to add QB.LIB. Do it in A) and then add every other file in B).
D) Now we have MYLIB.LIB containing DQB.LIB and MIDI.LIB. Now we have to create a suitable QLB file for the IDE (if we're gonna work from the IDE, if we don't, we don't need one as the LINKer only needs the LIB file). It is done using LINK:
LINK /Q MYLIB.LIB,MYLIB.QLB,NUL,BQLB45.LIB
Maybe you'll need to include QB full path before BQLB45.LIB as it is a QB file.
Now you have a new QLB and a new LIB files ready to be used.
#7 IT'S DONE!!
If you have comments or doubts please write. If there are obscure things you haven't found in every other tutorial and you'd like to know, write too and I'll try to include them in next releases of this tutorial.
Also, I'd like a 'correct english' version of this tutorial. If you wanna help, contact me.
I hope this has helped. Nathan.