morpheus.controllers.rss_controller.RSSController

class RSSController(feed_input, batch_size=128, run_indefinitely=None, enable_cache=False, cache_dir='./.cache/http', cooldown_interval=600, request_timeout=2.0)[source]

Bases: object

RSSController handles fetching and processing of RSS feed entries.

Parameters
feed_inputstr or list[str]

The URL or file path of the RSS feed.

batch_sizeint, optional, default = 128

Number of feed items to accumulate before creating a DataFrame.

run_indefinitelybool, optional

Whether to run the processing indefinitely. If set to True, the controller will continue fetching and processing If set to False, the controller will stop processing after the feed is fully fetched and processed. If not provided any value and if feed_input is of type URL, the controller will run indefinitely. Default is None.

enable_cachebool, optional, default = False

Enable caching of RSS feed request data.

cache_dirstr, optional, default = “./.cache/http”

Cache directory for storing RSS feed request data.

cooldown_intervalint, optional, default = 600

Cooldown interval in seconds if there is a failure in fetching or parsing the feed.

request_timeoutfloat, optional, default = 2.0

Request timeout in secs to fetch the feed.

Attributes
run_indefinitely

Property that determines to run the source indefinitely

session_exist

Property that indicates the existence of a session.

Methods

fetch_dataframes() Fetch and process RSS feed entries.
get_feed_stats(feed_url) Get feed input stats.
is_url(feed_input) Check if the provided input is a valid URL.
parse_feeds() Parse the RSS feed using the feedparser library.
fetch_dataframes()[source]

Fetch and process RSS feed entries.

Raises
Exception

If there is error fetching or processing feed entries.

get_feed_stats(feed_url)[source]

Get feed input stats.

Parameters
feed_urlstr

Feed URL that is part of feed_input passed to the constructor.

Returns
FeedStats

FeedStats instance for the given feed URL if it exists.

Raises
ValueError

If the feed URL is not found in the feed input provided to the constructor.

classmethod is_url(feed_input)[source]

Check if the provided input is a valid URL.

Parameters
feed_inputstr

The input string to be checked.

Returns
bool

True if the input is a valid URL, False otherwise.

parse_feeds()[source]

Parse the RSS feed using the feedparser library.

property run_indefinitely

Property that determines to run the source indefinitely

property session_exist: bool

Property that indicates the existence of a session.

Previous morpheus.controllers.rss_controller.FeedStats
Next morpheus.controllers.serialize_controller
© Copyright 2023, NVIDIA. Last updated on Feb 2, 2024.