Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 87 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,92 @@ ECMAScript interpreter for Cython & Python built for
- Being tiny and easy to use
- Having ECMA6 Support
- Having a maintained backend (QuickJS-NG)
- Being a good companion alongside [selectolax](https://github.com/rushter/selectolax).
- Being a good companion alongside [selectolax](https://github.com/rushter/selectolax) or beautiful-soup the choice is yours...
- License friendly, after abandoning pyduktape due to the backend no longer being maintained but also having a pretty poor license all together, It inspired me to try something new for a change that could run newer HTML5 Javascript for any puzzle that is thrown your way.

## Quick Example


```python
from cyjs import Context, Runtime


def main():
# You can also provide a runtime if needed it's usage before making multiple contexts however
# is completely optional
rt = Runtime()

ctx = Context(rt)

ctx.eval("function add(a, b){ return a + b; }; globalThis.add = add;")
add_func = ctx.get_global().get("add")

# 3
print(add_func(1, 2))

if __name__ == "__main__":
main()
```

Example of use with external html parser tools.

```python
from cyjs import Context
from selectolax.lexbor import LexborHTMLParser

# This example demonstates ways of cracking javascript out of webpages
# with an expernal HTML Parser and cyjs to handle the javascript logic.

# Know that A True HTML5 Dom-API Might require you to make your own
# functions and imagination but also reverse engineering the target page.

HTML_PAGE = b"""
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Document</title>
</head>
<body>
<div class="captcha">
<script>
function fake_captcha(name) {
return name + "-key";
}
</script>
<!-- Use your imagination a little... -->
</div>
<div>
<script>
this.captcha = fake_captcha('123')
</script>
</div>
</body>
</html>
"""

def main():
html = LexborHTMLParser(HTML_PAGE)
captcha = html.css_first("div.captcha > script").text(strip=True)

fake_solver = Context()
fake_solver.eval(captcha + "\nglobalThis.fake_captcha = fake_captcha;")
result = fake_solver.get_global().invoke("fake_captcha", "123")
# should print
# "123-key"
print(result)

if __name__ == "__main__":
main()

```

There will be more examples in a future update.




## NOTE
The code is still currently under construction as it's taking me a while to brainstorm how to best approch python to js value
converstions and vice versa. as well as how `Context` varaibles should work.

## Alternative Python Javascript Interpreters

Expand All @@ -38,18 +119,13 @@ for eatch to ensure it works correctly. I may be uploading this library to pypi

- [ ] If anybody finds a smarter approch to anything that has already been written throw me an issue or pull request.

- [ ] Reporting and fixing bugs.

- [ ] JSCFunction I haven't figured out a good solution for this one just yet since I'm trying to limit the number of cdef classes to keep the code small and easy to compile. We need a way to bind an opaque Python Object and trying the old quickjs method seems to trigger crashes (believe me when I say I tried doing that already).

- [ ] JSClass cdef class extension that can be subclassed in python and cython along with the hooks for all the JSClassExoticMethods (we need an approch to passing off a cdef class as an opaque value which I have not figured out how to do yet)
- [ ] JSClassFinalizer Hook
- [ ] JSClassGCMark Hook
- [ ] A safe approch for handling JSClass to python object conversion and vice versa (if possible)

- [ ] A Way to cancel a Promise Object which could lead to the creation of an __aiojs library__ extension built off this library for quickjs with callbacks from Promise to asyncio.Future while taking cancellation into consideration and adding Python Coroutine Support for quickjs-ng to be able figure out how to handle correctly.
- [ ] Maybe an easier way to pass off arguments from quickjs to python functions without making the code feel like we are fluking it.
- [ ] A Way to raise python exceptions from quickjs if possible.
- [ ] Better typehinting would be a bonus if someone could pull that off.
- [ ] A couple examples would be nice. If you need inspiration or an example yt-dlp-ejs might be a good freebie.
- [ ] C Extension Modules from python functions or modules have not been implemented yet due to the same problems as JSCFunction and JSClass being more or less puzzles to bind to python than they need to be.
- [ ] If anybody can figure out a way to make JS Arrays convert to any Python array or List (Preferrably array.array if the JS Array isn't typed but a list if it is typed) Feel free to figure this out for me.

3 changes: 3 additions & 0 deletions cyjs/__init__.pxd
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# cython: freethreading_compatible = True
# cython: language_level = 3
from ._cyjs cimport Context, Object, Runtime
25 changes: 24 additions & 1 deletion cyjs/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1,24 @@
from ._cyjs import * # noqa: F403
from ._cyjs import (
CancelledError,
Context,
JSError,
JSFunction,
Object,
Promise,
PromiseHookType,
Runtime,
)

__author__ = "Vizonex"
__version__ = "0.1.0"

__all__ = (
"CancelledError",
"Context",
"JSError",
"JSFunction",
"Object",
"Promise",
"PromiseHookType",
"Runtime",
)
2 changes: 1 addition & 1 deletion cyjs/_cyjs.pxd
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# cython: freethreading_compatible = True

# cython: language_level = 3
cimport cython
from libc.stdint cimport int64_t

Expand Down
76 changes: 35 additions & 41 deletions cyjs/_cyjs.pyi
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from collections.abc import Callable
from enum import IntEnum
from typing import Any, ParamSpec, TypeVar
from typing import Any, Generic, ParamSpec, TypeVar

_P = ParamSpec("_P")
_T = TypeVar("_T")
Expand Down Expand Up @@ -47,9 +47,6 @@ BEFORE: PromiseHookType = ...
AFTER: PromiseHookType = ...
RESOLVE: PromiseHookType = ...

# @cython.internal
# class PromiseHook: ...

class Runtime:
def __init__(self) -> None: ...
def compute_memory_usage(self) -> Any: ...
Expand All @@ -67,7 +64,10 @@ class Runtime:
def set_memory_limit(self, limit: int) -> Any: ...
def set_max_stack_size(self, max_stack_size: int) -> Any: ...
def update_statck_top(self) -> Any: ...
def set_promise_hook(self, func: object) -> object: ...
def set_promise_hook(
self,
func: Callable[["Context", PromiseHookType, Promise, Promise | None], None],
) -> object: ...

class _OView:
def __init__(self, obj: Object) -> None: ...
Expand Down Expand Up @@ -117,7 +117,7 @@ class Object:
# equivlent to context.eval_this(self)
def eval(
self,
code: object,
code: str | bytes | bytearray | memoryview,
filename: object = ...,
strict: bool = ...,
backtrace_barrier: bool = ...,
Expand All @@ -128,7 +128,7 @@ class Object:

def eval_module(
self,
code: object,
code: str | bytes | bytearray | memoryview,
filename: object = ...,
strict: bool = ...,
backtrace_barrier: bool = ...,
Expand All @@ -137,10 +137,10 @@ class Object:
"""evaluates javascript module code"""
...

class JSFunction(Callable[_P, _T]):
class JSFunction(Generic[_P, _T]):
context: Context

def __call__(self, *args:_P.args, **kwargs:_P.kwargs) -> _T: ...
def __call__(self, *args: _P.args, **kwargs: _P.kwargs) -> _T: ...
@property
def object(self) -> Object:
pass
Expand All @@ -164,10 +164,9 @@ class Context:
dom_exception: bool = ...,
promise: bool = ...,
) -> None: ...

def eval(
self,
code: object,
code: str | bytes,
filename: object = ...,
strict: bool = ...,
backtrace_barrier: bool = ...,
Expand All @@ -178,64 +177,61 @@ class Context:

def eval_module(
self,
code: object,
code: str | bytes,
filename: object = ...,
strict: bool = ...,
backtrace_barrier: bool = ...,
promise: bool = ...,
) -> object:
"""evaluates javascript module code"""
...


def get_global(self) -> Object:
...
def json_parse(self, json: object) -> Object:
...
def get_global(self) -> Object: ...
def json_parse(self, json: object) -> Object: ...
def get(self, name: bytes | str) -> Object | Any:
"""Implements a Shortcut for converting a global object to a python
"""Implements a Shortcut for converting a global object to a python
object and setting a value to utilize
off of."""
...

def set(self, name: bytes | str, item: object) -> None:
def set(
self, name: bytes | str, item: object | Object | Promise | JSFunction
) -> None:
"""Sets an item to the current globalThis object"""
...

def add_function(
self, func: Callable[...], name: str | bytes | None = None, magic: int = 11
) -> JSFunction:
"""adds a python function to quickjs using
`JS_NewCClosure` since Quickjs-ng doesn't have
self, func: Callable[_P, _T], name: str | bytes | None = None, magic: int = 11
) -> JSFunction[_P, _T]:
"""adds a python function to quickjs using
`JS_NewCClosure` since Quickjs-ng doesn't have
a good way for bidning python fucntions well yet
:param func: the python function to invoke with

:param func: the python function to invoke with
quickjs note: that it may not pass along keyword arguments `**kw`
:param name: an alternative name to give to the function being passed
:param magic: the magic value of the js function (Not much is known about it
:param magic: the magic value of the js function (Not much is known about it
at the moment... defaults to 11 which reflects quickjs's own tests)
"""

def eval_this(
self,
code: object,
code: str | bytes | bytearray | memoryview,
this: Object | Any,
filename: object = ...,
strict: bool = ...,
backtrace_barrier: bool = ...,
promise: bool = ...,
) -> Any:...

) -> Any: ...
def eval_this_with_module(
self,
code: object,
self,
code: str | bytes | bytearray | memoryview,
this: Object | Any,
filename: object = ...,
strict: bool = ...,
backtrace_barrier: bool = ...,
promise: bool = ...,
) -> Any:...

) -> Any: ...

class CancelledError(Exception):
"""Promise was rejected"""
Expand All @@ -248,23 +244,21 @@ class InvalidStateError(Exception):
...

class Promise(Object):
def add_done_callback(self, fn: object) -> object:
def add_done_callback(self, fn: Callable[["Promise"], None]) -> object:
"""Attaches a callable callback when promise finishes or raises an exception"""
...

def exception(self) -> object: ...
def exception(self) -> BaseException: ...
def done(self) -> bool: ...
def remove_done_callback(self, fn: object) -> int:
def remove_done_callback(self, fn: Callable[["Promise"], None]) -> int:
"""Remove all instances of a callback from the "call when done" list.

Returns the number of callbacks removed.
"""
...

def result(self) -> object: ...
def poll(self) -> object:
def result(self) -> Any: ...
def poll(self) -> Any:
"""Polls QuickJS Eventloop a single cycle while attempting
to wait for this Promise to complete"""
...


20 changes: 12 additions & 8 deletions cyjs/_cyjs.pyx
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
# cython: freethreading_compatible = True
from .quickjs cimport *
from cpython.bytes cimport PyBytes_FromStringAndSize
from cpython.unicode cimport PyUnicode_FromString, PyUnicode_FromStringAndSize
from cpython.exc cimport PyErr_NoMemory, PyErr_SetObject, PyErr_WriteUnraisable
from cpython.mem cimport PyMem_Malloc, PyMem_Realloc, PyMem_Free
from cpython.long cimport PyLong_FromString, PyLong_AsLongAndOverflow
from cpython.bool cimport PyBool_FromLong
from cpython.buffer cimport PyObject_CheckBuffer
from cpython.tuple cimport PyTuple_GET_SIZE
from cpython.bytes cimport PyBytes_FromStringAndSize
from cpython.exc cimport (PyErr_CheckSignals, PyErr_NoMemory, PyErr_Occurred,
PyErr_SetObject, PyErr_WriteUnraisable)
from cpython.list cimport PyList_AsTuple
from cpython.long cimport PyLong_AsLongAndOverflow, PyLong_FromString
from cpython.mem cimport PyMem_Free, PyMem_Malloc, PyMem_Realloc
from cpython.object cimport PyObject_CallObject, PyObject_Str
from cpython.exc cimport PyErr_CheckSignals, PyErr_Occurred
from cpython.tuple cimport PyTuple_GET_SIZE
from cpython.unicode cimport PyUnicode_FromString, PyUnicode_FromStringAndSize

from .quickjs cimport *


cdef extern from "Python.h":
# tweaked signature just a little for our purposes...
Expand All @@ -24,8 +26,10 @@ cdef extern from "Python.h":
cimport cython

# TODO: I'm writing a new cython utils library and a yyjson writer would be a good use-case here...

import json


cdef class JSError(Exception):
"""Represents Numerous Exceptions raised from CYJS"""

Expand Down
7 changes: 5 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[project]
name = "cyjs"
version = "0.1.0"
description = "Add your description here"
dynamic = ["version"]
description = "A Next Generation QuickJS Intrepreter for cython & python"
readme = "README.md"
authors = [
{ name = "Vizonex", email = "VizonexBusiness@gmail.com" }
Expand All @@ -21,3 +21,6 @@ requires = ["setuptools", "cython", "wheel"]
[tool.setuptools]
zip-safe = false
packages = ['cyjs']

[tool.setuptools.dynamic]
version = {attr = "cyjs.__version__"}
Loading
Loading