PhantomJS is a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.
The website(https://phantomjscloud.com) is awesome. But its free plan(500 Pages/Day) is not enough to use. So I find some open-source project which can deploy in Docker hosting or PaaS. For example, I will deploy b1nitp7iw/phantomjs-server in the Daocloud which offers deploy three docker app for free.
Deployment
Login your daocloud account. Use the keyword “b1nitp7iw” to search the image.
Click the image and be ready to deploy.
Choose to testing plan to deploy.
Set port to 8080.
Finally, you can see the app link on the dashboard.
Tesing
It will return nothing to directly run curl.
It’s ok to run curl with phantomjs-server.
Prevent App Sleep
Althougn the Daocloud offer free plan, the apps will stop after 24 hours. We can use the restart API to prevent sleeping. Before this,you should know the APP ID and API token.
Get your APP ID.
Get your API token.
Use the PostAgent of huginn to post the request to API and set the Schedule less than 24 hours.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
{ "post_url": "https://openapi.daocloud.io/v1/apps/2a7b3709-****-(APP ID)-****439e9bf1/actions/restart", "expected_receive_period_in_days": "1", "content_type": "form", "method": "post", "payload": { }, "headers": { "Authorization": "66h** (API Token) **79s4" }, "emit_events": "false", "no_merge": "false", "output_mode": "clean" } |
Usage
Send the following link by GET method and obtain the response.
http://ze*****omjs.daoapp.io/getDom?url=http://example.com
This is a example in huginn.
Flow:RSS Agent–>Website Agent–>Data Output Agent
RSS Agent(gain the article list):
|
1 2 3 4 5 |
{ "expected_update_period_in_days": "5", "clean": "false", "url": "http://ze*****omjs.daoapp.io/getDom?url=https://moeclub.org/rsslatest.xml" } |
Website Agent(extract the full text):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
{ "expected_update_period_in_days": "2", "url": "http://ze*****omjs.daoapp.io/getDom?url={{url}}", "type": "html", "mode": "on_change", "extract": { "content": { "css": ".kratos-post-content", "value": "." }, "title": { "css": ".kratos-entry-title", "value": "string(.)" } } } |
Data Output Agent(generate the final rss):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
{ "secrets": [ "moeclub" ], "expected_receive_period_in_days": 2, "template": { "title": "萌咖", "item": { "title": "{{title}}", "description": "{{content}}" } }, "ns_media": "true" } |
Similar Projects
These are based on PhantomJs but not as stable as the image (phantomjs-server).
https://github.com/vbauer/manet
https://github.com/Netflix/sketchy