GSoC Week 6

GSoC 2017: Sludge Engine Week 6

Week task conclusion

Generally, this 6th week of GSoC project was good: the first mid-term evaluation has been validated and Robin’s Rescue is playable on ScummVM (still having some bugs to fix, though).

Again, thanks to my mentors _sev(Eugene Sandulenko), t0by(Tobia Tesan) and all scummvm team members that has helped me on bug fixing during this week.

To make a brief conclusion about what we have and have not achieved for this week :

 

Tasks completed:

  1. Load headless png files : flag to skip signature check + modify image loader
  2. FIx bugs related to loading background after loading zbuffer
  3. Other minor bug fixes …

 

To resolve later:

  1. There is still a problem with game’s timer, which is slower than expected
  2. Characters’ animation are very slow as well
  3. The text color is not 100% right

What’s for next week: Save & Load System

Tasks for next week :

  1. Clear the rest of issues in Robin’s Rescue
  2. Get save & load system up

Some findings about sludge

Sludge engine use headless png files (without 8 byte signature) for sprites.

Advertisements

GSoC Week 5

GSoC 2017: Sludge Engine Week 5

Week task conclusion

Generally, this 5th week of GSoC project was going smoothly: The z-Buffer zone (multi-layer effect) has been reproduced and fall back detection has been added

Again, thanks to my mentors _sev(Eugene Sandulenko), t0by(Tobia Tesan) and all scummvm team members that has helped me on z-Buffer during this week.

To make a brief conclusion about what we have and have not achieved for this week :

Tasks completed:

  1. z-BufferZone : load z-Buffer sprites + reproduce multi-layer effects
  2. fall back detection for sludge games

 

To resolve later:

  1. There is a segmentation fault when Robin’s Rescue starts. It may result from the game logo reading in the initilization, which is not implemented yet

What’s for next week: Bug fixes in Robin’s Rescue

Tasks for next week :

  1. Fix the segmentation error in Robin’s Rescue
  2. Clear all bugs in Robin’s Rescue

Also, the first GSoc evaluation will take place next week.

Some findings about sludge

z-Buffer Zone

In sludge, z-Buffer zones are different areas of a scene which can obscure character graphics, such as foreground silhouettes, doorframes and furniture. Their display has been ordered by a y-value to create different image layers (up to 16).

The above room image has the following z-buffer image (3 layers) :

And characters will change the image layer it lies in, when its own y-value changes. To reproduce this effect in scummvm, we used an array of 16 Lists for different layers.

GSoC Week 4

GSoC 2017: Sludge Engine Week 4

Week task conclusion

Generally, this 3rd week of GSoC project was good: the input works and we are able to reallly “play” the game. Bugs are cleared for the game demo Verb Coins now and there are only z-Buffer stuff left to get it fully work.

Again, thanks to my mentors _sev(Eugene Sandulenko), t0by(Tobia Tesan) and all scummvm team members that has helped me on input and bug fixes during this week.

  

To make a brief conclusion about what we have and have not achieved for this week :

Tasks completed:

  1. game input
  2. freeze/unfreeze scene
  3. Add character interaction
  4. object inventory = dark screen effect + paste image to background

To resolve later:

  1. When setting opacity for transparent surface blitting, we need to modify the second parameter of TS_ARGB() instead of the first one, which is strange. Related line

What’s for next week: z-Buffer Image & new game

Tasks for next week :

  1. Get Z-Buffer up: it is used for different areas of a scene which can obscure character graphics, such as foreground silhouettes, doorframes and furniture.
  2. Add game fallback detection
  3. Add game Robin’s rescue and test

Some findings about sludge

Sludge input

Sludge input is using common sdl input functions

Freeze and unfreeze game scene

In sludge, whenever a dialog, reaction menu is popped up, the scene get frozen. The principle is that they make a screenshot of every thing on the background and copy interactive elements to a temporary variable and set them to null. Then, when we unfreeze, we restore them all.

GSoC Week 3

GSoC 2017: Sludge Engine Week 3

Week task conclusion

Generally, this 3rd week of GSoC project was good: milestone 1 is completed as we are able to display texts and play sounds(wav, ogg) now, although not all sound system works because of the need of implementing advanced module file decoders for XM, IT, S3M formats, which is not a small task at all. So we decided to leave it for later.

Again, thanks to my mentors _sev(Eugene Sandulenko), t0by(Tobia Tesan) and all scummvm team members that has helped me on texts and sounds during this week.

week-3-text.png

To make a brief conclusion about what we have and have not achieved for this week :

Tasks completed:

  1. Display texts in sludge
  2. Play wav/ogg sounds
  3. Loop wav/ogg sounds
  4. Loop a wav/ogg sound list

And :

  1. Some platform specific code removed, code compiles on all platforms now
  2. Replace sludge color conversion functions by scummvm PixelFormat functions

For later (They don’t have much effects for now):

  1. XM, IT, S3M decoders
  2. replace utf8 functions by common/ustr.cpp

 

A minor problem that may not have much effect:

  1. The way I implement “loop sound list’ doesn’t give the exactly same result for the particular case that several same sounds loop together at different time intervals. But I guess it’s something we’ll never really do in games.

What’s for next week: Cursor & Input

In order to have a playable game, we will move to input stuff next week :

  1. Get cursor displayed
  2. Replace sludge input event by scummvm ones, all of which are well covered

Some findings about sludge

How texts work

Every font is represented by a series of sprites for characters. Every sprite includes 2 parts:

  • the black background that gives a black border for texts: so all texts in sludge have black borders
  • a white overlay image, whose colors can be changed by shaders. This effect can be reproduced by changing input parameters of scummvm blit funtion

Screenshot from 2017-06-10 22-44-32.png

How audios work

There are 5 sound formats supported by sludge: .WAV, .OGG and .XM, .IT, .S3M. Every sound is played by a call in the game script:

  • playSound(…) for WAV and OGG
  • startMusic(…) for module sound formats : .XM, .IT, .S3M

We also have the possibility to loop a list of sounds in sludge, which is done by building a circled linked list beforehand and at the end-of-stream callback function, “the next sound” in the linked list is played.

GSoC Week 2

GSoC 2017: Sludge Engine Week 2

Week task conclusion

In general, the second week of GSoC project was quite good: all goals achieved one day in advance, though the work was not as easy as the first week. Again, a huge thank you to my mentors _sev(Eugene Sandulenko), t0by(Tobia Tesan) and all scummvm team members that has helped me on graphics during this week.

To make a brief conclusion about what we have and have not achieved for this week :

pasted image 1

Tasks completed:

  1. Get backdrop (background) reading and displaying work
  2. Get the sprite system up based on 1
  3. Get spritebank up to have animations and display transparent sprites

And :

  1. The segmentation fault caused by animation nullptr has been fixed
  2. scummvm can compile on windows

For later (They don’t have much effects for now):

  1. There are shaders used in graphics in scummvm waiting to be rewritten (normally we rewrite it by figuring out what they do and reproducing the same effects by c++ code)
  2. OOPifying the code should be something coming quite soon, as we still work with global variables now and it begins getting complicated. Maybe for next week or the week after if time permits

Some problems left to be solved :

  1. The code don’t compile for Mac yet

What’s for next week: Text & Sound

There is still some work before getting all sludge graphic features up, but I reckon that’s enough for now and we will add them later when we need. What we will move on to next week are:

  1. Get texts displayed (Graphics::FontManager and Font)
  2. Play sound (Audio classes)

And this makes the first game demo “Welcome” work completely. And we can see dialogues for “Verb coin”.

Problems encountered during week 2

As in this week, I was stuck several times with different problems (solved thanks to help from mentors and scummvm team members), I think it’s good to talk about them.

1. Graphics format

First of all, we need to determine which graphic format is used in Sludge to initGraphics.

As a result of lacking basic graphic knowledge, I was wrong about the graphic format because of the existence of palette structure in Sludge, which is actually used for transferring 256-color image (8-bit image) to 32-bit image.

There is a number of way to see the graphic format, for example:

In the graphic initialization code of the engine, there will be clues like :

if (SDL_SetVideoMode(realWinWidth, realWinHeight, 32, videoflags) == 0) {

The input parameter 32 shows that sludge is using 32 bit images.

Besides, if we can have access to specific image byte loading code like:

target = snapshotTexture + 4 * picWidth * y + x * 4;
target[0] = (byte)redValue(c);
target[1] = (byte)greenValue(c);
target[2] = (byte)blueValue(c);
target[3] = (byte)255;

We can clearly see that they are reading at an offset of 4 bytes, which also means that sludge uses 32-bit images.

Another funny mistake that I made during image loading is that I got wrong with PixelFormat and mixed up the channels, which gives an image like:

welcome_wrongwhich actually iswelcome_right

Just scrutinize the bit shift code of PixelFormat, there will be a clue.

2. Sprite loading error

pasted image 0

Another interesting problem is that I got distorted sprites at first when I tried to load them. We can see that all bytes of the sprites have been correctly read but to the wrong place. Normally, there is a sprite dimension error for such problems. It’s strongly suggested to log the sprite width and length of the image loading procedure to see if everything goes well.

In my case, sprites’ width and length were modified halfway by some improper manipulation.

3. Sprite flickering

I also met a sprite flickering problem due to multiple updateScreen() calls.

The entire screen is in practice updated when updateScreen() is called, so if there is image flicker then you’re presumably not producing the same frame consistently.

For instance, multiple updateScreen() calls are made for an actual frame, and some of those overwrite something that’s been previously rendered, then that could generate such errors, since you’re effectively producing as many frames as you have calls to updateScreen().

Some findings about sludge

  1. Graphics format used in Sludge is true color (32 bpp)
  2. Character animation is saved as a series of sprites and a looping sound
  3. Sludge has 3 versions of sprite image (version 0, 1,2 saved as 8-bit image, version 3 as png)

GSoC Week 1

GSoC 2017: Sludge Engine Week 1

Week task conclusion

In general, my first week of working for GSoC project was going smoothly. And a huge thank you to my mentors _sev(Eugene Sandulenko), t0by(Tobia Tesan) and all scummvm team members that has helped me during this week.

Due to my project plan, my task for this week was originally :

Task 1-1 Read game data file, initialization, timer , main_loop:

  1. Use Common:File to read and slice game data file to init game objects and get index of sources in data file
  2. Main loop: checkInput, playAnimation, handleInput, display, wait_frame
    1. Use TimerManager for timer
  3. Define macros and built-in functions

We have modified the plan, though, because it’s not to rewrite the whole engine bit by bit as I originally thought, but to add whole engine files at first and stub all the parts calling libraries and functions forbidden by scummvm, then gradually unstub them using scummvm functions, till we have the whole engine.

To make a brief conclusion about what we have achieved and changed for this week :

Achieved :

  1. Add all sludge engine files into the scummvm and make it compile under Linux
  2. Replace original data reading functions by Common::File/SeekabbleReadStream
  3. We are moving to make graphics work for sludge

For later (They don’t have much effects for now):

  1. Timer, input, …
  2. File writing stuff

Some problems left to be solved :

  1. There is an segmentation fault due to the incomplete data loading of image files and animations whenever animation or sprite variables are referenced, at not initialized yet.
  2. The code don’t compile yet for Mac or Windows at present

What’s for next week: Graphics

Generally, what we will do next week is :

  1. Get backdrop (background) reading and displaying work
  2. Get the sprite system up based on 1
  3. Get spritebank up to have animations

We have started a little on making graphics works, hopefully the “segmentation fault” could be fixed then if we would be able to load animations.

Some findings about sludge

How game data works

Find out how do they parse the game data file and try to adapt engine objects

Inner structure of .slg file:

A string here is composed of:

  • 2 bytes to indicate the string length
  • a series of chars for the string

A resource block is like: (same for text, sub, object, data)

  • Text is a string.
  • Sub contains functions used and defined by user
  • Object (items that can be put into inventory and combined and characters)
  • Data (image, audio, video)

In Sludge, we stock the beginning position of the index of resources (startOfObjectIndex, startOfDataIndex, startOfDataIndex) to access them.

Built-in function

In sludge, not only the events, but also all resources are integrated through built-in functions and everything except raw bitmaps and waveforms is handled through a constructor in the scripts. To take a simple script for example :

sub init () {

addOverlay (‘image.tga’, 0, 0);

playSound (‘tada.wav’);

pause (60);

quitGame ();

}

We can see that the background and sound are all added through built-in functions called in these game scripts.

The whole game interpreter is basing on a stack machine to work. For built-in functions as well. That is to say, when a built-in function is called, all its attributes will be pushed into a stack which will be pop() inside the function for using.

Git tips: keep your branch clean and clear for a PR

When contributing to a open source project, we should keep commits clean and clear in the pull request that we create. In this case, rebase, reset are important git operations that we’d better get familiar with.

Get to know about git-reset and git-rebase

git-reset

git-reset works with refs, on your working directory and the index, without touching any commit objects (or other objects).

git-rebase

git-rebase on the other hand is used to rewrite previously made commit objects. So if you want to rewrite the history, git-rebase is what you want. Note that you should never rewrite history that was pushed and was available to someone else, as rebasing rewrites the objects making them incompatible with the old objects, resulting in a mess for anyone else involved.

Some use case examples

Once we’ve seen how basic operations work in git, there are a number of things that we can achieve with them.

Update a fork without pushing merge commit

# fetch
$ git fetch upstream 

# rebase branch on top of upstream/master
$ git checkout myBranch git rebase upstream/master

# force a push to update
$ git push -f

Deal with local commits (unpushed)

Undo commit(s)

Undo last commit and abandon all the changes (never see them again)

$ git reset --hard HEAD~1

Undo last commit and keep all the changes (to be reedited later)

# leave the files alone
$ git reset HEAD~1

# leave the files and the index alone
$ git reset --soft HEAD~1

Deal with pushed commits (rewrite commit history)

ATTENTION, for all the operations below, don’t use them if someone has already pulled your change. Only for your personal repository. Else, we should use revert and create a relative commit for it.

# revert one single commit
$ git revert <commit_hash>

# revert a range of commits
# <oldest_commit_hash> is the oldest commit that we want to keep and won't be removed
$ git revert <oldest_commit_hash>..<latest_commit_hash>

Suppose we work on the branch myBranch and command git log gives:

commit eb6e1791690c2498891071ecc944bb381b413732
Author: AAA <github.com@ccc.com>  Date: ...
Commit 6

commit 8c9e5dbe40fe4dfd83aa1425c6a71c9b271d33e7
Author: AAA <aaa@gmail.com>  Date: ...
Commit 5

commit 53b0b82e8754e5e74dda2dedd9f7da726b068df7
Author: BBB <bbb@gmail.com>  Date: ...
Commit 4

commit 2a7cda649ebea7997d5a6ac74c23569f2b0b83d9
Author: CCC <ccc@scummvm.org>  Date: ...
Commit 3

commit c788642b72ea524d720e36b28bf9e87e7b06fd28
Author: DDD <github.com@ddd.com>  Date: ...
Commit 2

commit b7638982c2a7b619494108224e956a95bec88f08
Author: EEE <github@bbb.n>  Date: ...
Commit 1

Squash several commits

Squash last 3 commits:

# rewrite history for last 3 commits
$ git rebase -i HEAD~3

Then, text editor will show you:

pick eb6e179 Commit 6
pick 8c9e5db Commit 5
pick 53b0b82 Commit 4

Change it to:

# squash commit 4 5 into commit 6
pick eb6e179 Commit 6
squash 8c9e5db Commit 5
squash 53b0b82 Commit 4

Save and quit. Done.

Squash all commits in a branch into one commit :

$ git checkout myBranch
$ git reset $(git merge-base master myBranch)
$ git add -A
$ git commit -m "one commit on myBranch"

Squash commits 2 4 6 into one single commit (non consecutive commits):

# rewrite history for last 5 commits(2 3 4 5 6)
$ git rebase -i HEAD~5

Then, text editor will show us:

pick eb6e179 Commit 6
pick 8c9e5db Commit 5
pick 53b0b82 Commit 4
pick 2a7cda6 Commit 3
pick c788642 Commit 2

Reorder commits to be squashed together:

pick 8c9e5db Commit 5
pick 2a7cda6 Commit 3
pick eb6e179 Commit 6
squash 53b0b82 Commit 4
squash c788642 Commit 2

Save and quit. Done

Remove commit(s)

Remove last commit(s) :

# Get back to the last known good commit(Commit 4 for example, so 5 6 are removed)
$ git push -f origin 53b0b82e8754e5e74dda2dedd9f7da726b068df7:myBranch

Remove commits 2 4 6 (use rebase) :

# rewrite history for last 5 commits(2 3 4 5 6)
$ git rebase -i HEAD~5

Then, text editor will show us:

pick eb6e179 Commit 6
pick 8c9e5db Commit 5
pick 53b0b82 Commit 4
pick 2a7cda6 Commit 3
pick c788642 Commit 2 

Delete all the lines of unwanted commits:

pick 8c9e5db Commit 5
pick 2a7cda6 Commit 3

Save and quit. Done.

Remove commits 2 4 6 (use cherry-pick):

$ git checkout myBranch

# checkout the last usable commit (Commit 1)
$ git checkout b7638982c2a7b619494108224e956a95bec88f08

# create a new branch to work on
$ git checkout -b repair

# cherry pick commits that we want to keep (Commmit 3 5)
$ git cherry-pick 2a7cda649ebea7997d5a6ac74c23569f2b0b83d9
$ git cherry-pick 8c9e5dbe40fe4dfd83aa1425c6a71c9b271d33e7

# back to myBranch
$ git checkout myBranch

# reset myBranch to last usable commit (Commit 1)
$ git reset --hard b7638982c2a7b619494108224e956a95bec88f08

# merge our repair branch onto myBranch
$ git merge repair

# push to update
$ git push --hard origin myBranch

Continue reading “Git tips: keep your branch clean and clear for a PR”