Extracting Ren'Pys scripting language

[rudistoned]

TL:DR Rename the repository into RSL and the problem goes away.

Sorry about the bad documentation on my part 


I said I wanted to turn this Ren'Py Scripting Language into an importable python package. The name I chose for that package is RSL    (any suggestions for a better name are welcome)

Now, the problem you ran into is that you are free to choose how to call your mercurial repository when you clone it, so I can't control that. Having the package also be the repository folder also has its advantages though, because that way you can keep the repository directly in your project and switch revisions any time without having to copy the resulting python package around.

@relative imports
Any import starting with "from" is a relative import.

[DocClox]

OK, I'll check that out tonight. Got to go to work right now. (This "work" business doesn't half take up your time... )

[rudistoned]

Be sure to update your repository before working on it. I commited a bugfix today.

[DocClox]

OK, interesting. Thanks for uploading it. I had a look at extending the signal mechanism so I could update my own widget with text and speaker events, but didn't get very far. I need to read that dispatch module when I have more time and fewer interruptions

That said, am I right in thinking you plan on having an integrated stage and speech box, rather than sending signals back to arbitrary widgets?

About relative imports: I thought that, strictly speaking, they were the ones that used leading dots in the module name, as defined in PEP 0328? Which isn't to say that you can't import from a local package with "from", but that's not what I was talking about. All the by-the-by now, obviously

[rudistoned]

That said, am I right in thinking you plan on having an integrated stage and speech box, rather than sending signals back to arbitrary widgets?

Actually, no, I do want the scripting language to just send signals. That way it will be completely agnostic of the used GUI toolkit.
I am not against bundling a stage and speech box implemented in Qt with RSL, but I won't write that for the first "production" version of RSL, simply because it is not necessary to use the package.

@signals
I never read the code of the dispatch module, just the documentation.

@PEP328
Yeah, I read that and I also read a lot of other pages on imports and got confused. You are generally right in saying that relative imports always use leading dots. These are sometimes referred to as "explicit relative imports". However, in the latest Python versions before Python3, there is an exception to the leading dot rule. Consider this file system structure:

Code: [Select]

main.py
pypkg
    foo.py
    bar.py
If an "import bar" statement inside foo.py finds a module named foo in the package it is currently in, it will import that. That is sometimes referred to as an "implicit relative import".
This exception is especially bad because PEP328 says that "import <>" is always absolute.

Relative imports must always use from <> import; import <> is always absolute.

Update:
I just noticed that somewhere along the way I broke Ren'Py's global namespace (all .rpy files are executed in the same namespace). That's annoying.

[DocClox]

Actually, no, I do want the scripting language to just send signals. That way it will be completely agnostic of the used GUI toolkit.
I am not against bundling a stage and speech box implemented in Qt with RSL, but I won't write that for the first "production" version of RSL, simply because it is not necessary to use the package.

Good enough. I mean you know me - I was planning on writing my own widgetry anyway, so signals is by far my preferred approach.

I never read the code of the dispatch module, just the documentation.

That was going to be my first google of the day. Thanks for the link - it makes a lot more sense now

Yeah, I read that and I also read a lot of other pages on imports and got confused. You are generally right in saying that relative imports always use leading dots. These are sometimes referred to as "explicit relative imports".

I thought maybe I'd misunderstood something. Your grasp of Python is better than mine, so I wanted to make sure I hadn't missed something fundamental.

[rudistoned]

I'm really comfortable with Python now, but it's been years since I've read some parts of the docs and I've forgotten stuff and mix stuff up and so on and so forth. I'm sure everybody knows the feeling. I'm glad you corrected me about the imports. I've occasionally had problems with the import system in the past. On of the trickier aspects is that, probably by mixing relative with absolute imports, you can create two versions of the same class. If you then check instances of both versions with "isinstanceof", only one of both versions will show up as superclass.




Update:
Squashed the namespace-breaking bug, which I entirely introduced myself by speeding through this thing a little too fast. Phew! For a moment I thought this is going to be a serious bug.


For the moment, I'm having way to much fun testing this thing, writing little scripts that happily jump from label to label, from file to file and print silly messages to the console along the way 

[_rhetorik_]

Have you guys considered using something like pyV8 to make javascript the scripting language for the game? Or cefpython to embed a browser to make the ui an html file?


I guess modders would like to be able to change the game using just javascript and html/css. Pyv8 is the JS engine behind Chrome and Cef is the Chrome itself.

[DocClox]

Update:
Squashed the namespace-breaking bug, which I entirely introduced myself by speeding through this thing a little too fast. Phew! For a moment I thought this is going to be a serious bug.

For the moment, I'm having way to much fun testing this thing, writing little scripts that happily jump from label to label, from file to file and print silly messages to the console along the way 

Glad to hear it. I've got my storybox widget made, but I'm holding off hooking it up until I'm a little less wiped.

Have you guys considered using something like pyV8 to make javascript the scripting language for the game? Or cefpython to embed a browser to make the ui an html file?

Oddly enough, I was looking at the QScript interface during lunch today. It's definitely doable, although if Rudi gets RSL going full steam, that might be easier for all concerned.

Not so keen on chrome, as a platform. I did look at Webkit and  QWebView to make the whole thing HTML based, but I can't see that happening now. At least not for WMAC.

Javascript though, I'm keeping an open mind. If only because it might offer better sandboxing than running  arbitrary python embedded in girl and item packs

[rudistoned]

As for me and my game, which you can read more about in its subforum if you're interested:


What would I gain by running javascript from Python? Python itself is a powerful and simple scripting language, so if I wanted a programming language for game event scripts, I could just stick with Python, right?

For the GUI, both our games rely on Qt, an established C++ application framework offering, among other things, many advanced widgets. Qt GUIs can also be customized/modded using style sheets which allegedly are a lot like CSS.

I did a little work with Dojo quite a few years back and the experience I had was that GUI development with web tools was surprisingly more difficult than using desktop tools. My comparison to Dojo was GTK2, which has quite a few advanced widgets and Glade, a very useful graphical design tool. While working with Dojo, I had to code a lot of things GTK would have done for me.


Please don't think I'm bashing your suggestions. I really am curious to hear the answers to my questions, and since you brought it up I'm assuming you know more about javascript and html/css than I do.

[_rhetorik_]

What would I gain by running javascript from Python? Python itself is a powerful and simple scripting language, so if I wanted a programming language for game event scripts, I could just stick with Python, right?

That depends. Using JS instead of python just for scripting language isnt exactly an advantage although i believe JS is more common than python. The advantage here would be to use HTML+JS combo. That would give an easily customisable ui (HTML+CSS) that could even offer different logic (because HTML+JS is fine but HTML+Python .... isn't much).

For the GUI, both our games rely on Qt, an established C++ application framework offering, among other things, many advanced widgets. Qt GUIs can also be customized/modded using style sheets which allegedly are a lot like CSS.

You wouldn't need to recompile the ui (in c++ or precompiled python dist) or change the python source code. That is already a big advantage. Then it is easily moddable by anyone who can do a website so your community appreciates.

I did a little work with Dojo quite a few years back and the experience I had was that GUI development with web tools was surprisingly more difficult than using desktop tools. My comparison to Dojo was GTK2, which has quite a few advanced widgets and Glade, a very useful graphical design tool. While working with Dojo, I had to code a lot of things GTK would have done for me.

I don't agree that programming UIs is easier than using web tools ... No experience with Dojo though. But GTK is far from being a reference of succint code. At least in C/C++ ...

[DocClox]

That depends. Using JS instead of python just for scripting language isnt exactly an advantage although i believe JS is more common than python. The advantage here would be to use HTML+JS combo. That would give an easily customisable ui (HTML+CSS) that could even offer different logic (because HTML+JS is fine but HTML+Python .... isn't much).

Having used both, I'd far sooner develop in Python than in HTML and JS. Though I am tempted to develop in Python and make some parts of the game scriptable in JS. Python is known to be difficult to sandbox, and if I'm going to have a full fledged scripting language, I just as soon have one I can make secure.

Also, with PyQt we don't need to worry about cross browser issues, and quirks of CSS placement which is a bonus. But really, Python is just nice to work with.

[rudistoned]

HTML+JS [...] would give an easily customisable ui (HTML+CSS) [...]
You wouldn't need to recompile the ui (in c++ or precompiled python dist) or change the python source code.

GUI elements with no or limited dynamic functionality (simple drag and drop, basic signalling between widgets) can be created and modified in Qt using a nice graphical design tool. Modding does not get easier than requiring no coding experience whatsoever.
GUI elements created in Qt Designer can be either precompiled or loaded dynamically at runtime from XML files. The same thing is basically true for GTK2.
I do agree though that probably more people know how to write JS + HTML source code than Python + Qt source code.

I don't agree that programming UIs is easier than using web tools ... No experience with Dojo though. But GTK is far from being a reference of succint code. At least in C/C++ ...

That's exactly why I brought GTK2 up. In comparison to Qt4, GTK2 is IMHO quite clumsy, unwieldy, unpolished and has fewer powerful widgets. However, in comparison to Dojo from about 5 years back, GTK2 seemed to be elegant and very easy to work with. I should mention that I am using/have used both Qt and GTK via Python bindings, so I never had to deal with the intricacies of C/C++ development.
I definitely agree that GUI development with web tools is easier then GUI development in C/C++, regardless of toolkit used. However, IMHO the "problem" there is dealing with C/C++, which certainly are harder to work with than scripting languages. If you use a mature desktop toolkit like Qt and a friendly scripting language like Python, I really don't see how web tools could be easier to work with.
I am quite unfamiliar with modern web development, so I might be totally wrong here, but:
*) web tools are usually based on a client/server architecture, right? In a desktop application, all that communication just complicates things because you actually don't need a server.
*) web tools run in sandboxes. While that is great from a security standpoint, a desktop application is not directly exposed to potential attacks from the whole world every day, so it does not have to be that resilient. It is clear that increasing security always decreases ease of use, because security requires restrictions. Just look at the fun that is accessing the local file system from Flash for an example of what I'm talking about.
*) this might be a Dojo-specific problem: I remember that I had a hard time getting nice stack tracebacks on errors from java script. I had to use additional tools just to debug efficiently. Python throws a nice traceback on every error and comes with a useful debugger in the standard library.

Python is known to be difficult to sandbox, and if I'm going to have a full fledged scripting language, I just as soon have one I can make secure.

What is secure? A good answer is IMHO: at least as secure as I need it to be. I agree that Python is definitely unsecure, and executing arbitrary modder-generated code is probably as unsecure as it gets. However, lets get some perspective. We are talking about hobbyist desktop applications here. Do we really expect our users to exploit and harm each other? Do we need to protect them? After all, they are running a sex game written by anonymous strangers on their computers which they downloaded from some forum on the internet. On top of that, they are running modifications to that game, written by other anonymous strangers. How secure can their experience be if they are willing to do stuff like that?
If I use one of the sandboxes the Python community created to run modder-generated Python code in, wouldn't that be secure enough? The user base of our games here is about 6 orders of magnitude smaller than the user base of the internet. How big do you think are the odds that one of them has the skills and motivation to compromise a Python sandbox just so he/she can hit a few hundred people tops (not everybody will run every mod)?

Also, with PyQt we don't need to worry about cross browser issues, and quirks of CSS placement which is a bonus. But really, Python is just nice to work with.

Well, rather than dealing with cross-browser issues we are dealing with portability and packaging issues. I've heard cross browser issues got a lot better lately due to Microsoft catching up, so the difference between web and desktop might not be that big any more in this regard.
But really, Python is just nice to work with. 

[_rhetorik_]

Also, with PyQt we don't need to worry about cross browser issues, and quirks of CSS placement which is a bonus. But really, Python is just nice to work with.

You don't need to bother with that when you control the browser. It is always the same browser .... At most you would have problems with portability.

GUI elements with no or limited dynamic functionality (simple drag and drop, basic signalling between widgets) can be created and modified in Qt using a nice graphical design tool. Modding does not get easier than requiring no coding experience whatsoever.
GUI elements created in Qt Designer can be either precompiled or loaded dynamically at runtime from XML files. The same thing is basically true for GTK2.
I do agree though that probably more people know how to write JS + HTML source code than Python + Qt source code.

I dislike Qt from my C++ experience with it although it is impressive. IMHO Qt isn't C++ it is more of a dialect as it needs its own compiler (but that is an entirely different discussion).
I guess the main idea would be replacing Python + Qt + XML by HTML + JS which i believe is more community friendy. But, as always, both methods have their pros/cons.

I am quite unfamiliar with modern web development, so I might be totally wrong here, but:
*) web tools are usually based on a client/server architecture, right? In a desktop application, all that communication just complicates things because you actually don't need a server.
*) web tools run in sandboxes. While that is great from a security standpoint, a desktop application is not directly exposed to potential attacks from the whole world every day, so it does not have to be that resilient. It is clear that increasing security always decreases ease of use, because security requires restrictions. Just look at the fun that is accessing the local file system from Flash for an example of what I'm talking about.
*) this might be a Dojo-specific problem: I remember that I had a hard time getting nice stack tracebacks on errors from java script. I had to use additional tools just to debug efficiently. Python throws a nice traceback on every error and comes with a useful debugger in the standard library.

1) That depends. There are many "levels" of web developing. Client/Server is certainly one but that can be used also by desktop/mobile apps. In the specific case you can completely forget it. A simple HTML + JS page don't need to be in a server or you wouldnt be able to open an html file from your PC.


2) I don't think you need to bother with it here. Perhaps you guys got me wrong. The idea was to embed the browser in the application to access local html files (so that modders could just change the HTML+JS to change UI and Logic). No issue with security or client/server. Anyway all depends on what you would expose to the JS environment.


3) When you have code interfacing there is always problems to debug. It could be a JS problem, but imo Chrome does a nice job reporting it and helping debug code, or a python problem where you would have access to regular python tools.

[rudistoned]

I guess the main idea would be replacing Python + Qt + XML by HTML + JS which i believe is more community friendy.

As far as I can tell, the only argument you brought to support that is that more people are familiar with HTML + JS. I give you that, it is true. IMHO, that does not outweigh the advantages of Qt. You first need to create something that can attract a community before modder preferences become significant for your project
Btw, Java and maybe C/C++ are also known by more programmers than Python, but that does not make them better choices for this project.

1) That depends. There are many "levels" of web developing. Client/Server is certainly one but that can be used also by desktop/mobile apps. In the specific case you can completely forget it. A simple HTML + JS page don't need to be in a server or you wouldnt be able to open an html file from your PC.

I understand that there is no physical server involved in your suggestion. I was talking about the internal architecture of the toolkit. I remember that Dojo input widgets were sending their input to some kind of server object and display widgets got their data from that server object. Dojo basically modelled the physical setup of its typical use case (displaying a website on a client talking to a server) in its architecture. That seems very natural to me for a web toolkit, but it may get in the way when all you want to do is write a simple desktop application.

so that modders could just change the HTML+JS to change UI and Logic

Is the advantage you see that HTML + JS is usually distributed as source, while other languages are usually distributed in compiled form? If so: the same is possible with Python. I already do that for event scripts and I will probably do it for the whole application once I have fixed some errors I made during early development.