Pink Petal Games
Game Discussion => General Discussion & Download => Topic started by: rudistoned on November 20, 2013, 03:18:57 AM
-
I'd like to be able to use renpy-like scripts in games not run by renpy. For this reason I am looking into renpys code to see how tightly the parser for renpy scripts is interwoven with the rest of the engine.
This is what I got so far:
The sequence of code called while renpy is launched that leads to loading and parsing the renpy script files (.rpy) in very broad strokes:
# error handling loop
renpy.bootstrap.bootstrap()
# main function called from error handling loop
renpy.main.main()
# load all .rpy files, which in their entirety together form "the script"
renpy.script.Script()
"""
This class represents a Ren'Py script, which is parsed out of a
collection of script files. Once parsing and initial analysis is
complete, this object can be serialized out and loaded back in,
so it shouldn't change at all after that has happened.
@ivar namemap: A map from the name of an AST node to the AST node
itself. This is used for jumps, calls, and to find the current
node when loading back in a save. The names may be strings or
integers, strings being explicit names provided by the user, and
integers being names synthesised by renpy.
@ivar initcode: A list of priority, Node tuples that should be
executed in ascending priority order at init time.
@ivar all_stmts: A list of all statements, that have been found
in every file. Useful for lint, but tossed if lint is not performed
to save memory.
"""
AST: http://en.wikipedia.org/wiki/Abstract_syntax_tree (http://en.wikipedia.org/wiki/Abstract_syntax_tree)
All .rpy files are parsed at some point and their contents is represented by the Script class. Comments in renpy code calls this "the script".
"The script" is divided into nodes in an abstract syntax tree. Those nodes are defined in the renpy.ast module.
renpy.translation.Translator can sort nodes that require different treatment into different attributes. For example, python nodes go into the Translator.python dict and string nodes usable with StringTranslator go into the Translator.strings dict.
# find all files that will collectively form the script
renpy.script.Script.scan_script_files()
.rpy and .rpyc files go into renpy.script.Script.script_files
.rpym and .rpymc go into renpy.script.Script.module_files
# from renpy.script.Script.__init__ call Translator.chain_translates
As far as I can tell, Translator.chain_translates will do nothing because it "loops" over Translator.chain_worklist, which is initialized to an empty list. Translator.chain_worklist is filled when Script.load_script runs, which is obviously after the constructor of Script is done.
# from renpy.main.main call renpy.scrip.Script.load_script()
for each .rpy and .rpyc file:
Run it through a series of loading methods, ending up at Script.load_file_core,
which calls renpy.parser.parse
# the parser for the renpy script language: renpy.parser
# This module contains the parser for the Ren'Py script language. It's
# called when parsing is necessary, and creates an AST from the script.
The parser module does not import anything not in Pythons stdlib directly. It uses some parts of renpy 23 times by calling renpy.<something> (7 times renpy.ast and 4 times renpy.atl, which we probably need anyway; 4 times renpy.config, which is easy to replace).
Bottom line: renpy.parser seems easy to extract if renpy.ast and renpy.atl are easy to extract too, which I have not looked into yet.
Update1:
renpy.ast contains the classes implementing all the statements of renpys scripting language. Several of these statements make good use of objects outside of the renpy.ast module. In total, there are 130 calls to renpy.<something>, most of which we would have to replace (not all because we probably don't need all statements).
renpy.atl refers to the Ren'Py Animation and Transformation language. Since that is specific to the GUI toolkit used, I don't think we need it, at least not for the initial version of our event scripting language.
In conclusion, it appears as if renpy.parser and renpy.ast contain most of the code we need. Taking that, removing everything we don't need and replacing all calls to renpys infrastructure with a simpler, generic environment is IMHO likely to be faster than implementing something equally powerful on our own. The renpy scripting language has been around for a while and received its share of bugfixes and performance improvements. I think we could benefit from that.
-
Interesting, I've been meaning to look under the Ren'Pys hood for a while but right now I don't even have time to work on PyTFall properly, please keep this thread updated :)
-
If not, I've got a couple of ideas on how we might render encounter trees using Python syntax, while keeping things reasonably clear. I'll back to hold off posting until I get back home tonight, but I think we can do this fairly cleanly.
-
Added an update to the opening post. Since this looks better than I expected, I'll start ripping pieces out of renpy and see if I can revive them afterwards ???
-
and after I've been having such fun here, too!
Seriously, that sounds like great need. What I've got is workable, but renpy's syntax is much nicer to work with :)
-
Sorry to crash your party :D
You know, this was A LOT easier than I expected. I can already parse the renpy demo script (the_question) and get a list of abstract syntax tree instances as return value. 8)
Makes me think. I read somewhere that every time new technology works the first time you try somewhere 1000 kittens die a horribly death to keep the universe in balance ???
Poor kittens...
I'm recording the changes I made, starting from the original renpy files, in a mercurial repository. I could upload it to source forge if you want to take a look at it.
-
that sounds good :)
[edit]
Gah! I just spent an hour arguing with gmail that had become convinced that I had disabled cookies. I'm starting to mutter "bloody Google" in the same tones that I used to say "bloody Microsoft!"
Anyway... This is what I was doodling during my lunch hour.
encounter_list = {}
class Encounter:
def __iadd__(self, enc_list):
name = enc_list[0]
encounter_list[name] = enc_list
return self
def __enter__(self):
return self
def __exit__(self, type, value, traceback):
print "type = ", type
print "value = ", value
#
# these are just so we can write chooser rather than "chooser"
#
# there are issues with namespace pollution, but I don't anticpate encounter
# definitions being mixed much with normal code, so probably not a
# problem
#
name = "name"
speaker = "speaker"
setting = "setting"
lines = "lines"
choose = "choose"
func = "func"
thumb = "thumb"
encounter_end = "encounter_end"
#
# the "with" construct is just to provide us with a block
# so we can define functions that we can later reference
# in the encounter list
#
with Encounter() as enc:
def counter_offer(index):
if index == 0:
show_encounter("best of luck")
if index == 1:
if player.is_female():
show_encounter("you too")
else:
show_encounter("forty and my protection")
if index ==2:
show_encounter("you should learn to haggle")
def choose_you_too(index):
# "Forty then, and you make do with just the girls",
# "I don't see that I have much choice...",
# "Sounds like a win-win scenario to me!"
if index == 0:
show_encounter("bah!")
if index == 1:
show_encounter("harsh business realities")
if index ==2:
show_encounter("such enthusiasm")
enc += [
"Hamid raises a concern",
{ speaker : "Hamid",
thumb : "hamid_thumb.png",
setting : "hamids_tent.png",
lines : [
"What are you doing buying slavegirls?",
"I said you could stay with me until you got your licence",
"I didn't say you should start building a harem!"
]
},
{
lines : [
"You to explain that the arrangement is only temporary\n"
"Just until you can get proper business premises.",
"Hamid seems unconvinced."
]
},
{ speaker : "Hamid",
thumb : "hamid_thumb.png",
lines : [
"What? I can not believe my ears!",
"This is my home! You cannot turn it into a brothel!"
]
},
{ lines : "A calculating look crosses Hamid's face"
},
{ speaker : "Hamid",
thumb : "hamid_thumb.png",
lines : [
"If I run the House, I get to fuck all the girls for free, right?",
"And fifty percent of the take!"
]
},
{ choose : [
"Forget it! I don't need a business partner.",
"Fifty percent? Try twenty!",
"All right, you have a deal!",
],
func : counter_offer
}
]
enc += [
"best of luck",
{ speaker : "Hamid",
thumb : "hamid_thumb.png",
setting : "hamids_tent.png",
lines : [
"And I do not need such an ungracious guest "
"who would take such advantage of his host's generosity",
"",
"Do you think you will fare better sleeping in Element Square?",
"You are welcome to try!"
]
},
{
lines : [
"Thinking of what you have seen of the Souk, you are forced to agree",
"Reluctantly, you accept Hamid's offer"
]
},
{ speaker : "Hamid",
thumb : "hamid_thumb.png",
setting : "hamids_tent.png",
lines : [
"Ha! I knew you would see sense!"
]
},
encounter_end
enc += [
"you too",
{ speaker : "Hamid",
thumb : "hamid_thumb.png",
setting : "hamids_tent.png",
lines : [
"I'll go as low as thirty, but for that I want you in my bed "
"as well as any of your girls whenever I want them"
]
},
{ choose : [
"Forty then, and you make do with just the girls",
"I don't see that I have much choice...",
"Sounds like a win-win scenario to me!"
],
func : choose_you_too
}
]
enc += [
"you should learn to haggle",
{ speaker : "Hamid",
thumb : "hamid_thumb.png",
setting : "hamids_tent.png",
lines : [
"We have a deal then! Though if we are to be partners "
"we shall have to work on your haggling skills!",
"No! Do not bother to thank me! it is all part of the service "
"now we are partners."
]
},
encounter_end
enc += [
"forty and my protection",
{ speaker : "Hamid",
thumb : "hamid_thumb.png",
setting : "hamids_tent.png",
lines : [
"Forty percent is as low as I will go.",
"But to sweeten the deal, I will offer my local expertise "
"as well as letting it be known that your girls are under "
"my protection",
"Take it or leave it, that's the best offer you'll get"
]
},
{ lines : [
"You have a feeling this is as good as it's going to get",
"You think back on the conditions in the Souk at night",
"It seems unlikely that you can do anything without some sort of base",
"",
"Reluctantly, you accept Hamid's offer",
]
},
{ speaker : "Hamid",
thumb : "hamid_thumb.png",
setting : "hamids_tent.png",
lines : [
"Excellent! Let us drink to a long and profitable friendship!"
]
},
encounter_end
]
enc += [
"bah!",
{ speaker : "Hamid",
thumb : "hamid_thumb.png",
setting : "hamids_tent.png",
lines : [
"Bah! It is not good for a woman to raise a man's hopes like that!",
"Still, I should not grumble. If this enterprise goes well "
"I shall have all the pussy a man could dream of!",
]
},
{
lines : "Hamid begins to unbuckle his belt"
},
{ speaker : "Hamid",
thumb : "hamid_thumb.png",
lines : [
"Speaking of which, I think it is time I conducted some "
"basic quality control. You may watch if that is what you wish"
]
},
{ lines : "And so began my partnership with Hamid..."
},
encounter_end
]
enc += [
"harsh business realities",
{ speaker : "Hamid",
thumb : "hamid_thumb.png",
setting : "hamids_tent.png",
lines : [
"It is good to meet a woman who understands the harsh realities of business! ",
"Far too many of you think only with your quim ... when you bother to think at all",
"Now I want you to get naked and break in your new slave",
"I shall watch and assess her technique ... and yours",
"",
"And don't give me that face! This is going to be fun! You'll see!"
]
},
{ lines : "And so began your association with Hamid..."
},
encounter_end
]
enc += [
"such enthusiasm",
{ speaker : "Hamid",
thumb : "hamid_thumb.png",
setting : "hamids_tent.png",
lines : [
"Such enthusiasm! Have you ever consider you may be more suited to the workforce "
"than to management?",
"No matter! No matter! Let us seal the deal!"
]
},
{
lines : "Hamid begins to unbuckle his belt"
},
{ speaker : "Hamid",
thumb : "hamid_thumb.png",
lines : [
"Now I want you and your new slave naked and on my bed, right now!",
"I intend to satisfy both slave and mistress many times before the night is done"
]
},
{ lines : "And so began your friendship with Hamid..."
},
encounter_end
]
Which is probably as sane a way to do it as we'll find using vanilla python. That said, I still like the idea of Ren'Py's parser better.
i was going to work out the back end for all that, but there's not much point at the moment, so I'll go back to making widgets for a bit :)
-
I took a two day break from renpy, now I'm back at work again.
Turned out (of course...) that I could get the earlier code to run so easily because it basically skipped from start to finish without doing most of the work. Nevertheless, I filled in the pieces in between and I can now execute successfully labels and python code blocks. Scene and With statements are handled by printing a message for now. Next up: getting dialog lines to work.
What I can tell already is that this will be a "fork", not a "mod". Maintaining the delta and copying patches from newer versions of renpy will be difficult and might be too much trouble for their worth. I don't think that's a big problem though. Most of the code consists of features we don't need (rollback) or performance optimizations I deactivated to simplify things.
@DocCloxLooks interesting, thanks for posting it :)
-
Sounds very promising.
I agree, we don't need renpy as a whole, just the scene player and DSL in module form. Hell, I'd settle for just the parser :)
-
My goal is to be able to parse a renpy script and execute it. If during execution something happens that is outside the scope of this scripting language module, it should emit some kind of signal, e.g. "show-image" and give a path. In the first iteration it will just print "show image", imagepath.
The dialogue handling code is integrated into character management, which uses all kinds of systems. When this thing is done, it will be one big, hairy mess of code that has no real meaning any more, interspersed with rare bits and pieces that actually get the job done. In any case, it will be A LOT smaller then renpy and it should be relatively easy to reduce it further from there.
UPDATE:
This is how far I've got. First, the renpy script:
# Declare images used by this game.
image bg lecturehall = "lecturehall.jpg"
image bg uni = "uni.jpg"
image bg meadow = "meadow.jpg"
image bg club = "club.jpg"
image sylvie normal = "sylvie_normal.png"
image sylvie giggle = "sylvie_giggle.png"
image sylvie smile = "sylvie_smile.png"
image sylvie surprised = "sylvie_surprised.png"
image sylvie2 normal = "sylvie2_normal.png"
image sylvie2 giggle = "sylvie2_giggle.png"
image sylvie2 smile = "sylvie2_smile.png"
image sylvie2 surprised = "sylvie2_surprised.png"
# Declare characters used by this game.
define s = Character('Sylvie', color="#c8ffc8")
define m = Character('Me', color="#c8c8ff")
# The game starts here.
label start:
$ bl_game = False
#play music "illurock.ogg"
python:
# test some python code
r = range(10)
print "Counting to 10"
for i in r: print i
scene bg lecturehall
with fade
"Well, professor Eileen's lecture was interesting."
"But to be honest, I couldn't concentrate on it very much."
"I had a lot of other thoughts on my mind."
"And they all ended up with a question."
"A question, I've been meaning to ask someone."
scene bg uni
with fade
"When we came out of the university, I saw her."
show sylvie normal
with dissolve
"She was a wonderful person."
"I've known her ever since we were children."
"And she's always been a good friend."
"But..."
"Recently..."
"I think..."
"... that I wanted more."
"More just talking... more than just walking home together when our classes ended."
"And I decided..."
menu:
"... to ask her right away.":
jump rightaway
"... to ask her later.":
jump later
label rightaway:
show sylvie smile
s "Oh, hi, do we walk home together?"
And here is the output I get from the thing I have assembled here from parts of renpy:
Counting to 10
0
1
2
3
4
5
6
7
8
9
TODO [scene] display (u'bg', u'lecturehall') at layer master
TODO [with] fade transition
TODO [say] None Well, professor Eileen's lecture was interesting. {'interact': True, 'cb_args': {}}
TODO [say] None But to be honest, I couldn't concentrate on it very much. {'interact': True, 'cb_args': {}}
TODO [say] None I had a lot of other thoughts on my mind. {'interact': True, 'cb_args': {}}
TODO [say] None And they all ended up with a question. {'interact': True, 'cb_args': {}}
TODO [say] None A question, I've been meaning to ask someone. {'interact': True, 'cb_args': {}}
TODO [scene] display (u'bg', u'uni') at layer master
TODO [with] fade transition
TODO [say] None When we came out of the university, I saw her. {'interact': True, 'cb_args': {}}
TODO [show] (u'sylvie', u'normal')
TODO [with] dissolve transition
TODO [say] None She was a wonderful person. {'interact': True, 'cb_args': {}}
TODO [say] None I've known her ever since we were children. {'interact': True, 'cb_args': {}}
TODO [say] None And she's always been a good friend. {'interact': True, 'cb_args': {}}
TODO [say] None But... {'interact': True, 'cb_args': {}}
TODO [say] None Recently... {'interact': True, 'cb_args': {}}
TODO [say] None I think... {'interact': True, 'cb_args': {}}
TODO [say] None ... that I wanted more. {'interact': True, 'cb_args': {}}
TODO [say] None More just talking... more than just walking home together when our classes ended. {'interact': True, 'cb_args': {}}
TODO [say] None And I decided... {'interact': True, 'cb_args': {}}
TODO [menu] display choices [(u'... to ask her right away.', 'True', 0), (u'... to ask her later.', 'True', 1)] None
TODO [show] (u'sylvie', u'smile')
It then throws an exception complaining that it does not know who "s" is. Might be because I start execution at the start label, so the character and image definitions may have never run. Will have to look how renpy does this.
UPDATE2:
Character and image definitions work now. Menus/choices are currently ignored and the execution code always selects the first choice to continue. I'm hoping that's because I disabled the code in the menu statement and not because the block linking code does something it should not.
UPDATE3:
Menu statements are working now. The application using this scripting language receives a signal containing a list of choices and a callback method. It then selects a choice and hands it over to the callback, thus telling the script where to continue. Signals are transmitted via a pure-python signalling package.
-
Cool! There can't be much left to do from the sound of it.
I'm starting to look forward to having a play with this. I've got the into that I wrote for the abortive ren'py version of AC. it might be fun r to see if I can get that working :D
-
Well, to get it working in the most basic sense, there is not much left to do. Basically, I need to add a few lines that emit a signal to the implementation of every statement that does something outside the scope of the scripting language itself (show an image, register a character, ...). That should be doable really fast.
Then, the scripting language should be refactored into an importable python package, so you don't have to have its files inside your main program directory. That was the last thing I was working on and even though it should have been easy, the whole thing stopped working after I made those changes.
It's not the first time Python's import system is messing with me. I suspect that the changed import statements now create new instances of the modules they import, instead of always importing the same instance (like the difference between "from foo import FooClass" and "import foo.FooClass as FooClass"). I don't have time to look into that today though. Maybe tomorrow.
I could always upload it somewhere if you want to take a look on it yourself.
-
That sounds good. It'll need to wait until I get home, but I should get some time tonight, all things being equal :)
-
For simplicities sake, I uploaded the code into a new mercurial repository in my Pytherworld project. Here's the link to the Ren'Py Scripting Language (https://sourceforge.net/p/pytherworld/rsl/ci/default/tree/). The example directory contains the first draft for a simple example application that demonstrates how to incorporate the RSL package into an application.
-
Got it, thanks.
Had a quick stab at running the example - it seems to be missing a module called RSL - Ren'Py Script Language I assume rather than Remote Service Library. I can't find a file or folder with RSL in the name anywhere in the repo so I'm guessing that either the file(s) didn't get added in, or possibly the top level files need to be in a folder called RSL ... although __init__.py seems to suggest otherwise.
Anyway, it's getting towards my bedtime, further investigation will need to wait. I've only had time for a quick glance at the code, so I'm probably missing all sorts of obvious here.
Anyway, more once I've had some sleep :)
[edit]
Actually, imports like RSL.Config and RSL.signals would probably work if config.py and signals were in a folder called RSL. Or maybe using relative imports, if I could just think a bit straighter. So that's probably part of what sprang loose when you repackaged it as standalone.
Not sure about RSL.base tho.
Anyway, sleep...
-
TL:DR Rename the repository into RSL and the problem goes away.
Sorry about the bad documentation on my part :(
I said I wanted to turn this Ren'Py Scripting Language into an importable python package. The name I chose for that package is RSL :D (any suggestions for a better name are welcome)
Now, the problem you ran into is that you are free to choose how to call your mercurial repository when you clone it, so I can't control that. Having the package also be the repository folder also has its advantages though, because that way you can keep the repository directly in your project and switch revisions any time without having to copy the resulting python package around.
@relative imports
Any import starting with "from" is a relative import.
-
OK, I'll check that out tonight. Got to go to work right now. (This "work" business doesn't half take up your time... ;))
-
Be sure to update your repository before working on it. I commited a bugfix today.
-
OK, interesting. Thanks for uploading it. I had a look at extending the signal mechanism so I could update my own widget with text and speaker events, but didn't get very far. I need to read that dispatch module when I have more time and fewer interruptions :)
That said, am I right in thinking you plan on having an integrated stage and speech box, rather than sending signals back to arbitrary widgets?
About relative imports: I thought that, strictly speaking, they were the ones that used leading dots in the module name, as defined in PEP 0328 (http://www.python.org/dev/peps/pep-0328/)? Which isn't to say that you can't import from a local package with "from", but that's not what I was talking about. All the by-the-by now, obviously :)
-
That said, am I right in thinking you plan on having an integrated stage and speech box, rather than sending signals back to arbitrary widgets?
Actually, no, I do want the scripting language to just send signals. That way it will be completely agnostic of the used GUI toolkit.
I am not against bundling a stage and speech box implemented in Qt with RSL, but I won't write that for the first "production" version of RSL, simply because it is not necessary to use the package.
@signals
I never read the code of the dispatch module, just the documentation. (https://pypi.python.org/pypi/dispatcher)
@PEP328
Yeah, I read that and I also read a lot of other pages on imports and got confused. You are generally right in saying that relative imports always use leading dots. These are sometimes referred to as "explicit relative imports". However, in the latest Python versions before Python3, there is an exception to the leading dot rule. Consider this file system structure:
main.py
pypkg
foo.py
bar.py
If an "import bar" statement inside foo.py finds a module named foo in the package it is currently in, it will import that. That is sometimes referred to as an "implicit relative import".
This exception is especially bad because PEP328 says that "import <>" is always absolute.
Relative imports must always use from <> import; import <> is always absolute.
Update:
I just noticed that somewhere along the way I broke Ren'Py's global namespace (all .rpy files are executed in the same namespace). That's annoying.
-
Actually, no, I do want the scripting language to just send signals. That way it will be completely agnostic of the used GUI toolkit.
I am not against bundling a stage and speech box implemented in Qt with RSL, but I won't write that for the first "production" version of RSL, simply because it is not necessary to use the package.
Good enough. I mean you know me - I was planning on writing my own widgetry anyway, so signals is by far my preferred approach.
I never read the code of the dispatch module, just the documentation. (https://pypi.python.org/pypi/dispatcher)
That was going to be my first google of the day. Thanks for the link - it makes a lot more sense now :)
Yeah, I read that and I also read a lot of other pages on imports and got confused. You are generally right in saying that relative imports always use leading dots. These are sometimes referred to as "explicit relative imports".
I thought maybe I'd misunderstood something. Your grasp of Python is better than mine, so I wanted to make sure I hadn't missed something fundamental.
-
I'm really comfortable with Python now, but it's been years since I've read some parts of the docs and I've forgotten stuff and mix stuff up and so on and so forth. I'm sure everybody knows the feeling. I'm glad you corrected me about the imports. I've occasionally had problems with the import system in the past. On of the trickier aspects is that, probably by mixing relative with absolute imports, you can create two versions of the same class. If you then check instances of both versions with "isinstanceof", only one of both versions will show up as superclass.
Update:
Squashed the namespace-breaking bug, which I entirely introduced myself by speeding through this thing a little too fast. Phew! For a moment I thought this is going to be a serious bug.
For the moment, I'm having way to much fun testing this thing, writing little scripts that happily jump from label to label, from file to file and print silly messages to the console along the way ;D
-
Have you guys considered using something like pyV8 to make javascript the scripting language for the game? Or cefpython to embed a browser to make the ui an html file?
I guess modders would like to be able to change the game using just javascript and html/css. Pyv8 is the JS engine behind Chrome and Cef is the Chrome itself.
-
Update:
Squashed the namespace-breaking bug, which I entirely introduced myself by speeding through this thing a little too fast. Phew! For a moment I thought this is going to be a serious bug.
For the moment, I'm having way to much fun testing this thing, writing little scripts that happily jump from label to label, from file to file and print silly messages to the console along the way ;D
Glad to hear it. I've got my storybox widget made, but I'm holding off hooking it up until I'm a little less wiped.
Have you guys considered using something like pyV8 to make javascript the scripting language for the game? Or cefpython to embed a browser to make the ui an html file?
Oddly enough, I was looking at the QScript interface during lunch today. It's definitely doable, although if Rudi gets RSL going full steam, that might be easier for all concerned.
Not so keen on chrome, as a platform. I did look at Webkit and QWebView to make the whole thing HTML based, but I can't see that happening now. At least not for WMAC.
Javascript though, I'm keeping an open mind. If only because it might offer better sandboxing than running arbitrary python embedded in girl and item packs ;)
-
As for me and my game, which you can read more about in its subforum (http://forum.otherworldgame.net/index.php?board=9.0) if you're interested:
What would I gain by running javascript from Python? Python itself is a powerful and simple scripting language, so if I wanted a programming language for game event scripts, I could just stick with Python, right?
For the GUI, both our games rely on Qt, an established C++ application framework offering, among other things, many advanced widgets. Qt GUIs can also be customized/modded using style sheets which allegedly are a lot like CSS.
I did a little work with Dojo quite a few years back and the experience I had was that GUI development with web tools was surprisingly more difficult than using desktop tools. My comparison to Dojo was GTK2, which has quite a few advanced widgets and Glade, a very useful graphical design tool. While working with Dojo, I had to code a lot of things GTK would have done for me.
Please don't think I'm bashing your suggestions. I really am curious to hear the answers to my questions, and since you brought it up I'm assuming you know more about javascript and html/css than I do.
-
What would I gain by running javascript from Python? Python itself is a powerful and simple scripting language, so if I wanted a programming language for game event scripts, I could just stick with Python, right?
That depends. Using JS instead of python just for scripting language isnt exactly an advantage although i believe JS is more common than python. The advantage here would be to use HTML+JS combo. That would give an easily customisable ui (HTML+CSS) that could even offer different logic (because HTML+JS is fine but HTML+Python .... isn't much).
For the GUI, both our games rely on Qt, an established C++ application framework offering, among other things, many advanced widgets. Qt GUIs can also be customized/modded using style sheets which allegedly are a lot like CSS.
You wouldn't need to recompile the ui (in c++ or precompiled python dist) or change the python source code. That is already a big advantage. Then it is easily moddable by anyone who can do a website so your community appreciates.
I did a little work with Dojo quite a few years back and the experience I had was that GUI development with web tools was surprisingly more difficult than using desktop tools. My comparison to Dojo was GTK2, which has quite a few advanced widgets and Glade, a very useful graphical design tool. While working with Dojo, I had to code a lot of things GTK would have done for me.
I don't agree that programming UIs is easier than using web tools ... No experience with Dojo though. But GTK is far from being a reference of succint code. At least in C/C++ ...
-
That depends. Using JS instead of python just for scripting language isnt exactly an advantage although i believe JS is more common than python. The advantage here would be to use HTML+JS combo. That would give an easily customisable ui (HTML+CSS) that could even offer different logic (because HTML+JS is fine but HTML+Python .... isn't much).
Having used both, I'd far sooner develop in Python than in HTML and JS. Though I am tempted to develop in Python and make some parts of the game scriptable in JS. Python is known to be difficult to sandbox, and if I'm going to have a full fledged scripting language, I just as soon have one I can make secure.
Also, with PyQt we don't need to worry about cross browser issues, and quirks of CSS placement which is a bonus. But really, Python is just nice to work with.
-
HTML+JS [...] would give an easily customisable ui (HTML+CSS) [...]
You wouldn't need to recompile the ui (in c++ or precompiled python dist) or change the python source code.
GUI elements with no or limited dynamic functionality (simple drag and drop, basic signalling between widgets) can be created and modified in Qt using a nice graphical design tool. Modding does not get easier than requiring no coding experience whatsoever.
GUI elements created in Qt Designer can be either precompiled or loaded dynamically at runtime from XML files. The same thing is basically true for GTK2.
I do agree though that probably more people know how to write JS + HTML source code than Python + Qt source code.
I don't agree that programming UIs is easier than using web tools ... No experience with Dojo though. But GTK is far from being a reference of succint code. At least in C/C++ ...
That's exactly why I brought GTK2 up. In comparison to Qt4, GTK2 is IMHO quite clumsy, unwieldy, unpolished and has fewer powerful widgets. However, in comparison to Dojo from about 5 years back, GTK2 seemed to be elegant and very easy to work with. I should mention that I am using/have used both Qt and GTK via Python bindings, so I never had to deal with the intricacies of C/C++ development.
I definitely agree that GUI development with web tools is easier then GUI development in C/C++, regardless of toolkit used. However, IMHO the "problem" there is dealing with C/C++, which certainly are harder to work with than scripting languages. If you use a mature desktop toolkit like Qt and a friendly scripting language like Python, I really don't see how web tools could be easier to work with.
I am quite unfamiliar with modern web development, so I might be totally wrong here, but:
*) web tools are usually based on a client/server architecture, right? In a desktop application, all that communication just complicates things because you actually don't need a server.
*) web tools run in sandboxes. While that is great from a security standpoint, a desktop application is not directly exposed to potential attacks from the whole world every day, so it does not have to be that resilient. It is clear that increasing security always decreases ease of use, because security requires restrictions. Just look at the fun that is accessing the local file system from Flash for an example of what I'm talking about.
*) this might be a Dojo-specific problem: I remember that I had a hard time getting nice stack tracebacks on errors from java script. I had to use additional tools just to debug efficiently. Python throws a nice traceback on every error and comes with a useful debugger in the standard library.
Python is known to be difficult to sandbox, and if I'm going to have a full fledged scripting language, I just as soon have one I can make secure.
What is secure? A good answer is IMHO: at least as secure as I need it to be. I agree that Python is definitely unsecure, and executing arbitrary modder-generated code is probably as unsecure as it gets. However, lets get some perspective. We are talking about hobbyist desktop applications here. Do we really expect our users to exploit and harm each other? Do we need to protect them? After all, they are running a sex game written by anonymous strangers on their computers which they downloaded from some forum on the internet. On top of that, they are running modifications to that game, written by other anonymous strangers. How secure can their experience be if they are willing to do stuff like that?
If I use one of the sandboxes the Python community created to run modder-generated Python code in, wouldn't that be secure enough? The user base of our games here is about 6 orders of magnitude smaller than the user base of the internet. How big do you think are the odds that one of them has the skills and motivation to compromise a Python sandbox just so he/she can hit a few hundred people tops (not everybody will run every mod)?
Also, with PyQt we don't need to worry about cross browser issues, and quirks of CSS placement which is a bonus. But really, Python is just nice to work with.
Well, rather than dealing with cross-browser issues we are dealing with portability and packaging issues. I've heard cross browser issues got a lot better lately due to Microsoft catching up, so the difference between web and desktop might not be that big any more in this regard.
But really, Python is just nice to work with. ;D
-
Also, with PyQt we don't need to worry about cross browser issues, and quirks of CSS placement which is a bonus. But really, Python is just nice to work with.
You don't need to bother with that when you control the browser. It is always the same browser .... At most you would have problems with portability.
GUI elements with no or limited dynamic functionality (simple drag and drop, basic signalling between widgets) can be created and modified in Qt using a nice graphical design tool. Modding does not get easier than requiring no coding experience whatsoever.
GUI elements created in Qt Designer can be either precompiled or loaded dynamically at runtime from XML files. The same thing is basically true for GTK2.
I do agree though that probably more people know how to write JS + HTML source code than Python + Qt source code.
I dislike Qt from my C++ experience with it although it is impressive. IMHO Qt isn't C++ it is more of a dialect as it needs its own compiler (but that is an entirely different discussion).
I guess the main idea would be replacing Python + Qt + XML by HTML + JS which i believe is more community friendy. But, as always, both methods have their pros/cons.
I am quite unfamiliar with modern web development, so I might be totally wrong here, but:
*) web tools are usually based on a client/server architecture, right? In a desktop application, all that communication just complicates things because you actually don't need a server.
*) web tools run in sandboxes. While that is great from a security standpoint, a desktop application is not directly exposed to potential attacks from the whole world every day, so it does not have to be that resilient. It is clear that increasing security always decreases ease of use, because security requires restrictions. Just look at the fun that is accessing the local file system from Flash for an example of what I'm talking about.
*) this might be a Dojo-specific problem: I remember that I had a hard time getting nice stack tracebacks on errors from java script. I had to use additional tools just to debug efficiently. Python throws a nice traceback on every error and comes with a useful debugger in the standard library.
1) That depends. There are many "levels" of web developing. Client/Server is certainly one but that can be used also by desktop/mobile apps. In the specific case you can completely forget it. A simple HTML + JS page don't need to be in a server or you wouldnt be able to open an html file from your PC.
2) I don't think you need to bother with it here. Perhaps you guys got me wrong. The idea was to embed the browser in the application to access local html files (so that modders could just change the HTML+JS to change UI and Logic). No issue with security or client/server. Anyway all depends on what you would expose to the JS environment.
3) When you have code interfacing there is always problems to debug. It could be a JS problem, but imo Chrome does a nice job reporting it and helping debug code, or a python problem where you would have access to regular python tools.
-
I guess the main idea would be replacing Python + Qt + XML by HTML + JS which i believe is more community friendy.
As far as I can tell, the only argument you brought to support that is that more people are familiar with HTML + JS. I give you that, it is true. IMHO, that does not outweigh the advantages of Qt. You first need to create something that can attract a community before modder preferences become significant for your project ;)
Btw, Java and maybe C/C++ are also known by more programmers than Python, but that does not make them better choices for this project.
1) That depends. There are many "levels" of web developing. Client/Server is certainly one but that can be used also by desktop/mobile apps. In the specific case you can completely forget it. A simple HTML + JS page don't need to be in a server or you wouldnt be able to open an html file from your PC.
I understand that there is no physical server involved in your suggestion. I was talking about the internal architecture of the toolkit. I remember that Dojo input widgets were sending their input to some kind of server object and display widgets got their data from that server object. Dojo basically modelled the physical setup of its typical use case (displaying a website on a client talking to a server) in its architecture. That seems very natural to me for a web toolkit, but it may get in the way when all you want to do is write a simple desktop application.
so that modders could just change the HTML+JS to change UI and Logic
Is the advantage you see that HTML + JS is usually distributed as source, while other languages are usually distributed in compiled form? If so: the same is possible with Python. I already do that for event scripts and I will probably do it for the whole application once I have fixed some errors I made during early development.
-
As far as I can tell, the only argument you brought to support that is that more people are familiar with HTML + JS. I give you that, it is true. IMHO, that does not outweigh the advantages of Qt. You first need to create something that can attract a community before modder preferences become significant for your project ;)
You don't need to get rid of Qt. In fact you would still need a background gui toolkit (Qt,Wx, Gtk, ...) to make the window and the browser ui to it. The HTML would be just the game ui. In C++ you can do what i am suggesting in Qt using QtWebView probably the same using python. The replacing i mentioned was just for modding and the "in-game" ui.
I undertand very little about Dojo, but it is a web thing isnt it? There is nothing about client/server in this architecture. If you really want to see some client/server in it you can think of JS scripting as a client and python application that embeds the browser as a server ... But i think that is a little way off.
Is the advantage you see that HTML + JS is usually distributed as source, while other languages are usually distributed in compiled form? If so: the same is possible with Python. I already do that for event scripts and I will probably do it for the whole application once I have fixed some errors I made during early development.
Yes, that is good. But if i give you a text in aramaic it won't be that helpful, right? (Assuming that you doesn't know how to read aramaic...). There is a difference between changing the game code and changing the game resources.
You say that changing the game is easy because it is python and i am saying that changing the game resources is more friendly. This way the ui can be made a resource and the game code can be kept as a single version.
But anyway, i ve seen the game progress and it is looking really nice :)
Hoping to play it soon .... Whatever way you use to make it !!
-
Thanks! Although I should probably point out that there are two developers for two different games here. I'm making wmac and rudi's project is Pytherworld.
I can't see me changing the Dev platform now. I did consider qwebview and is - for Perth much the reasons you outline. it's a bit late to be changing things at that level though. which isn't to day I won't include some sort of modder friendly scripting. I'm still pondering on the pros and cons of that one. (sending this from my phone BTW so please excuse any weirdness from the auto complete :))
@rudi: regarding security, use case I have in mind is scripting girl encounters as in vanilla wm. the potential attack vector is if some unscrupulous individual posts a girl pack and scripts it with a malware downloader or spam relay or some such.
obviously I'll check anything I distribute, but I can't check everything. so I'd sooner not make it easy for the had guys to subvert my game.
so yeah, sandboxing is a consideration.
which I'd about the only advantage of JavaScript over Python or rps. but I think it is an important one.
-
But if i give you a text in aramaic it won't be that helpful, right? [...] There is a difference between changing the game code and changing the game resources.
Declaring files containing code as "game ressources" instead of "game code" does not make it easier to modify them at all ;) In both cases you need to be able to understand the code.
Any javascript coder should be able to understand the following example code from wikipedia (http://en.wikipedia.org/wiki/JavaScript#More_advanced_example):
function LCMCalculator(x, y) {
var checkInt = function (x) {
if (x % 1 !== 0) {
throw new TypeError(x + " is not an integer");
}
return x;
};
this.a = checkInt(x)
this.b = checkInt(y);
}
You are telling me the following Python code would look like aramaic to him? Really? I'm not buying it.
class LCMCalculator(object):
def __init__(self, x, y):
def check_int(x):
if x % 1 != 0:
raise TypeError(str(x) + " is not an integer")
return x
self.a = check_int(x)
self.b = check_int(y)
That code even uses language features I've never needed nor used in my projects (nested functions)! A lot of code will be simpler to understand than that.
This way the ui can be made a resource and the game code can be kept as a single version.
That has more to do with separating the game logic from the UI code than it has to do with the language the code is written in, don't you think?
@DocClox
Okay, but do you really think plugging one hole will stop a criminal individual? I don't know all that much about IT security, but usually there are a lot of different ways you can attack any software. What if the malicious code is not in the code part of the girlpack, but embedded within one of the images? What if the criminal just provides a mod that changes the Python part of the application? What if the attack is not delivered via a mod but via some sort of utility, like a girl pack builder/viewer? Or a launcher that customizes parts of the game people want do mod but you chose to keep away from them? I honestly don't think you can get a hobbyist game into a secure state.
-
Rudi, I can't even secure my house!
There are people who could pick the lock on my front door, there are pick guns and diamond drills... if the bad guys want in, they'll get in.
Nevertheless, I still lock my door at night.
I can't make the game bullet proof, but I still feel I have a duty to take reasonable precautions.
[edit]
Corrected some of the atrocities committed by my phone's predictive text function...
-
Declaring files containing code as "game ressources" instead of "game code" does not make it easier to modify them at all ;) In both cases you need to be able to understand the code.
That has more to do with separating the game logic from the UI code than it has to do with the language the code is written in, don't you think?
From the Dev perspective i slightly disagree. You can separate logic and ui in your code. If you consider your UI as a resource, you dont need to "recompile" if you are working on a compiled "environment". On a script, the difference still holds although more subtle.
From a user perspective the diference is big. Your user don't need to sniff through your code to find the UI and make changes. It can just edit an extern file. Btw, Whoremaster does it with XML files and had to implement its own "browser" using SDL.
-
Btw, Whoremaster does it with XML files and had to implement its own "browser" using SDL.
I should probably declare an interest at this juncture and point out that I wrote the XML processing code for WM, including the XML screen layout code.
Oddly enough, I can remember thinking at the time that it would all be easier with a proper HTML parser and javascript integration. Which is probably what got me thinking about Lua scripting. The problem with Lua and Wm wasn't so much a lack of HTML for layouts as it was the work entailed in exposing the underlying game. That (among other things) started me thinking about Clonemaster which would have coded primarily in Lua using the bones of necno's SDL as game the engine. And from there, via Daisy's Otherworld and an extended flirtation with Skyrim, it let me back here to Python and WMAC.
Odd how theses things progress, sometimes :)
-
I fully agree that having a GUI that is defined in ressource files (like XML) and dynamically loaded at runtime is great. I did that back when I was using GTK and it worked great. Qt can load its GUI from XML too, but I ran into some problems when I tried it, switched to the precompiled variant and never looked back. I might try it again when I seriously start working on Pytherworlds GUI.
I still don't get why reading code in HTML files is less "sniffing through code" than reading code in the ui packages of my application. In both cases the GUI code is located in several files within several directories.
@DocClox
That's exactly my point :) . You can't make it bulletproof anyway, so trying to make it somewhat secure can happen when the game is more or less done.
To be honest, I lock my door at night because it makes me feel better, not because I think it makes a difference. Anyone who wants to break in won't be deterred by a locked door. People who do not want to break in don't need a locked door to stay out.
-
To be honest, I lock my door at night because it makes me feel better, not because I think it makes a difference. Anyone who wants to break in won't be deterred by a locked door. People who do not want to break in don't need a locked door to stay out.
The point is that if you make it a little bit difficult, 99% of would be intruders will go and look for easier pickings. Conversely, if you leave it wide open, people may wander in who wouldn't normally consider doing so. There's a world of difference between "wide open" and "reasonably secure". I'd like to avoid "wide open", personally.
I don't think there are any hard and fast rules here. I'm not saying everyone should worry about sandboxing user mods. But I'm going to keep it as a consideration, for the reasons I've outlined.
-
I see the difference between "wide open" and "reasonably secure". I just think slowing down development for a gain in security is bad for the project, at least until there is a nice, playable beta. All hobbyist projects run the risk of never getting to a stable, post-beta state, so development speed is IMHO paramount.
Update:
Btw, _rhetorik_, are you aware of Brothelsim? I think that game uses non-Flash web technology.
-
I see the difference between "wide open" and "reasonably secure". I just think slowing down development for a gain in security is bad for the project, at least until there is a nice, playable beta. All hobbyist projects run the risk of never getting to a stable, post-beta state, so development speed is IMHO paramount.
Oh hell yes. I'm not even going to write any test code until I have the game playable. Third party scripts for individual girls is one of the last things I'll look at. I only mentioned it because it is one area (possibly the only area) where I can see a benefit from embedding Javascript in PyQt. First things first :)
-
I fully agree that having a GUI that is defined in ressource files (like XML) and dynamically loaded at runtime is great. I did that back when I was using GTK and it worked great. Qt can load its GUI from XML too, but I ran into some problems when I tried it, switched to the precompiled variant and never looked back. I might try it again when I seriously start working on Pytherworlds GUI.
What is the differece of a XML gui and an HTML gui? The HTML gui is "free"(as in little developer time) the browser already works and it already comes with an scripting language (which is lacking in XML unless you code it yourself afaik). Not sure why you like the XML gui as a resource but you don't like the HTML gui as a resource...
I still don't get why reading code in HTML files is less "sniffing through code" than reading code in the ui packages of my application. In both cases the GUI code is located in several files within several directories.
You just need 1 folder with 10~20 html files. Perhaps another set of .js and a .css file. No inheritance, no need to search through hierarquies where that thing was defined ... nothing too hard for non-developers. Nowadays people make some increadibly powerful gui using HTML5 and libraries like YUI and jQuery and all that is available for "free" (again as in little developer time).
-
What is the differece of a XML gui and an HTML gui?
The widgets described in the XML are the difference ;)
I like the possibilities Qt gives me: Powerful widgets, GUI design with a polished graphical tool, internationalization support and many other features I have not used so far, but which are there should my requirements grow.
I just do not believe that web tools have caught up so much that they can play in the same league as desktop tools as far as desktop development is concerned.
The HTML gui is "free"(as in little developer time) the browser already works and it already comes with an scripting language (which is lacking in XML unless you code it yourself afaik).
I highly doubt that the GUI of a desktop application can be written faster in HTML than in Qt, so Qt GUIs are at least as "free" as HTML GUIs. Qt works too, just as the browser, so why mention it? Python is also a scripting language, just as JavaScript, so why mention it?
The scripting language this thread is about has very little to do with coding GUI logic. The Ren'Py scripting language is not equivalent to JavaScript, not by a long shot. It is useful to script dialog-heavy ingame events and was originally developed for visual novels. It is not a full-fledged programming language.
Not sure why you like the XML gui as a resource but you don't like the HTML gui as a resource...
I like both for the fact that you can alter the GUI without altering the code of the application. What I dislike about HTML is that I strongly suspect that its usefulness for desktop GUIs is not up to par with Qt.
You just need 1 folder with 10~20 html files. Perhaps another set of .js and a .css file. No inheritance, no need to search through hierarquies where that thing was defined ... nothing too hard for non-developers.
Technically, I could develop my whole application in a single source file, using only functional programming and builtin classes. No custom classes, no inheritance, only the global namespace and and the local namespaces of the functions. According to your argument that should be really easy to understand, right? Well, it's not. It's easier to see were every variable is defined, sure, but what are they for? Organizing your code in class hierarchies is useful for a lot of reasons and is such a widespread practice that I don't feel I have to explain it.
Placing all source files into the same folder does not make it easier to understand what they do. Again, structure helps to convey meaning.
What kind of GUI modifications are you talking about that you think a non-developer could do with HTML?
Nowadays people make some increadibly powerful gui using HTML5 and libraries like YUI and jQuery and all that is available for "free" (again as in little developer time).
Could you point to some good examples of such GUIs, so I can look at them myself?
Oh hell yes. I'm not even going to write any test code until I have the game playable. Third party scripts for individual girls is one of the last things I'll look at. I only mentioned it because it is one area (possibly the only area) where I can see a benefit from embedding Javascript in PyQt. First things first :)
I agree with that completely :)
-
The widgets described in the XML are the difference ;)
I like the possibilities Qt gives me: Powerful widgets, GUI design with a polished graphical tool, internationalization support and many other features I have not used so far, but which are there should my requirements grow.
I just do not believe that web tools have caught up so much that they can play in the same league as desktop tools as far as desktop development is concerned.
This site is a google project to explore HTML5. This a tutorial about defining custom tags in HTML.
http://www.html5rocks.com/en/tutorials/webcomponents/customelements/ (http://www.html5rocks.com/en/tutorials/webcomponents/customelements/)
Using QtWebView you could add QtWidgets to the web page. They might still support it.
I highly doubt that the GUI of a desktop application can be written faster in HTML than in Qt, so Qt GUIs are at least as "free" as HTML GUIs. Qt works too, just as the browser, so why mention it? Python is also a scripting language, just as JavaScript, so why mention it?
The scripting language this thread is about has very little to do with coding GUI logic. The Ren'Py scripting language is not equivalent to JavaScript, not by a long shot. It is useful to script dialog-heavy ingame events and was originally developed for visual novels. It is not a full-fledged programming language.
I like both for the fact that you can alter the GUI without altering the code of the application. What I dislike about HTML is that I strongly suspect that its usefulness for desktop GUIs is not up to par with Qt.
Python doesn't run from an HTML file. If you can change the logic in the GUI lvl than you can change the entire logic of the app.
There are many companies/applications migrating from traditional ui to this model. Steam and Mozilla Songbird are just a browser with HTML and App Logic as a resource. In fact i guess all the mozilla stuff runs inside a Firefox box (not sure though).
http://html5-demos.appspot.com/static/gdd11-modern-web-apps/index.html#1 (http://html5-demos.appspot.com/static/gdd11-modern-web-apps/index.html#1)
http://www.html5rocks.com/en/business/ (http://www.html5rocks.com/en/business/)
What kind of GUI modifications are you talking about that you think a non-developer could do with HTML?
Could you point to some good examples of such GUIs, so I can look at them myself?
I expect i non-devs can try their hand on HTML, perhaps even on some basic JS. But i don't expect them to try their hand on python even though it shares some similarities with JS.
http://www.hongkiat.com/blog/html5-web-applications/ (http://www.hongkiat.com/blog/html5-web-applications/)
-
I expect i non-devs can try their hand on HTML, perhaps even on some basic JS. But i don't expect them to try their hand on python even though it shares some similarities with JS.
Well, that's one point on which we disagree. To be perfectly honest, this is the last time I am willing to explain the obvious.
I don't see how basic Python differs from basic JavaScript in difficulty. If anything, Python is easier because it needs less syntactic clutter. Just look at the simple examples from wikipedia:
# JS
var y = 2;
# Python
y = 2
# JS
console.log("Hello world!");
# Python
print "Hello world!"
# JS
function factorial(n) {
if (n === 0) {
return 1;
}
return n * factorial(n - 1);
}
# Python
def factorial(n):
if n == 0:
return 1
return n * factorial(n - 1)
Thank you for the example applications though :)
I will have a look at them later.
Btw, Steam: Whoa, that thing annoyed the crap out of me because it just would not work and I needed it for a game I wanted to play. I spent hours fiddling around with it, just so it would run on a regular 64bit Windows 7 machine. Then it ran once and the next day it was broken again. That's what you get for paying for your games ::)
It got better last year though, installing and updating Steam works for my computers now.
-
Lurching back on topic for a moment ;) I've posted a demo of my story widget on the WMAC thread.
It uses a list driven approach like the one I posted earlier rather than RPS. That doesn't rule out RPS in the future, since all I need is a way to deliver events in sequence, and frankly, what I have at the moment is ugly-ugly-ugly!
The source is in the archive, an exe also.