morpheus.controllers.rss_controller.RSSController

Bases: object

RSSController handles fetching and processing of RSS feed entries.

Parameters

feed_input

The URL or file path of the RSS feed.

batch_size

Number of feed items to accumulate before creating a DataFrame.

run_indefinitely

Whether to run the processing indefinitely. If set to True, the controller will continue fetching and processing If set to False, the controller will stop processing after the feed is fully fetched and processed. If not provided any value and if feed_input is of type URL, the controller will run indefinitely. Default is None.

enable_cache

Enable caching of RSS feed request data.

cache_dir

Cache directory for storing RSS feed request data.

cooldown_interval

Cooldown interval in seconds if there is a failure in fetching or parsing the feed.

request_timeout

Request timeout in secs to fetch the feed.

strip_markup

When true, strip HTML & XML markup from the from the content, summary and title fields.

Attributes

run_indefinitely

Property that determines to run the source indefinitely

Methods

fetch_dataframes()[source]

Fetch and process RSS feed entries.

Raises

Exception

If there is error fetching or processing feed entries.

get_feed_stats(feed_url)[source]

Get feed url stats.

Parameters

feed_url

Feed URL that is part of feed_input passed to the constructor.

Returns

FeedStats

FeedStats instance for the given feed URL if it exists.

Raises

ValueError

If the feed URL is not found in the feed url provided to the constructor.

classmethod is_url(feed_input)[source]

Check if the provided url is a valid URL.

Parameters

feed_input

The url string to be checked.

Returns

bool

True if the url is a valid URL, False otherwise.

parse_feeds()[source]

Parse the RSS feed using the feedparser library.

property run_indefinitely

Property that determines to run the source indefinitely